RFC: TestCase events for Tester

Milo

Sometimes we use in our tests some additional tools (Mockista, mockery, database connection, PHP server, memcached, dummy presenter, different container configurations, …). Current solution is to create some base TestCases with different combinations of these tools. But it get complicated as more and more tools used. Not mentioning you need to call all of them in tearDown and setUp correctly.

We think about solution inspired by Codeception modules. What we need to create something like that is ability of modules to hook on some TestCAse lifecycle events.

Co we suggest to add to TestCase these events which can be hooked from modules to extends TestCAse behaviour:

beforeSetUp
afterSetUp
beforeTearDown
afterTearDown
beforeRunTest
afterRunTest

Usage for Mockery in some module then can look like:

class MockeryModule {

  public static function register(TestCase $testCase) {
    $testCase->afterTearDown[] = function($testCase) {
      Mockery::close();
    }
  }

}

Or for database extension:

class DatabaseModule {

  public static function register(TestCase $testCase) {
    $testCase->beforeSetUp[] = function($testCase) {
      $this->database->connect();
      $this->database->importData();
    }
    $testCase->afterTearDown[] = function($testCase) {
      $this->database->clearData();
    }
  }

}

Concept of modules need to be think throught and is not ready yet. It probably should be addon. But adding this events is first step to be able to do this, it is not BC break and can improve extendability of TestCase.

Proposed solution in PR #246.

Milo

I agree that some hooks can be useful. But IMHO, it should be done in a less error prone way. Public properties are to much benevolent.

First I had in my mind is:

$testCase->addListener(TestCase::BEFORE_SETUP, function (TestCase $tc) {

});

Felix

I vote for listeners too. “Nette like” events are useful, but …

mystik

Listener is ok I think. But if we use Listener how about addListener(ITestCaseListener $listener) more like standard Listener pattern.

Then we should decide if use single ITestCaseListener with all events or separate interface for each event like IOnBeforeSetupListener

Last edited by mystik (2015-08-14 22:33)

Milo

mystik wrote:

Listener is ok I think. But if we use Listener how about addListener(ITestCaseListener $listener) more like standard Listener pattern.

That's a next possibility.

Then we should decide if use single ITestCaseListener with all events or separate interface for each event like IOnBeforeSetupListener

Separated interfaces (in this case) are error prone:

interface IOnBeforeSetupListener {}
interface IOnAfterSetupListener {}

class Listener implements IOnBeforeSetupListener # and here I forgot next interface
{
}

mystik

ok I will prepare PR with single interface

mystik

next question: should we have onTestPass and onTestFail in this interface for additional extension?

Milo

mystik wrote:

ok I will prepare PR with single interface

Please don't, save your time. This is just discussion for now.

mystik

Proposed solution:

<?php
interface ITestCaseListener {

  public function onBeforeSetUp(TestCase $testCase, $testName, $args);
  public function onAfterSetUp(TestCase $testCase, $testName, $args);

  public function onBeforeTearDown(TestCase $testCase, $testName, $args);
  public function onAfterTearDown(TestCase $testCase, $testName, $args);

  public function onBeforeRunTest(TestCase $testCase, $testName);
  public function onAfterRunTest(TestCase $testCase, $testName);

  public function onBeforeRun(TestCase $testCase);
  public function onAfterRun(TestCase $testCase);

  public function onTestPass(TestCase $testCase, $testName, $args);
  public function onTestFail(TestCase $testCase, $testName, $args, $exception);
}
?>

<?php
class TestCase {

  // ...

  public function addListener(ITestCaseListener $listener) {
    // ...
  }

  // ...
}
?>

Milo

Such interface has problem with possible BC. Adding new event means BC break for existing implementors.

Beside event system. What about traits?

trait TDatabase
{
	private $database;

	function initDatabase() {...}
	function cleanDatabase() {...}
}

class Test extends TestCase
{
	use TDatabase;

	function setUp()
	{
		self::initDatabase();
		parent::setUp();
	}

	...
}

Pros for traits:

order of calls as you wish
access to private properties

Cons for traits:

more coding
missing constructor

mystik

Traits would entirely kill original purpose of this RFC of simplifing extensions of tests. Such extensisons would be limited to traits and cannost use inheritance of its own, fluent interface, multiple use of single extension (two database connections), …

Possible BC break of interface can be solved in two ways:

Separate interfaces

<?php
interface ITestCaseListener {

}

interface OnBeforeSetUpListener extends ITestCaseListener {
  public function onBeforeSetUp(TestCase $testCase, $testName, $args);
}

interface OnBeforeSetUpListener extends ITestCaseListener {
  public function onBeforeSetUp(TestCase $testCase, $testName, $args);
}

// ...

?>

<?php
class DatabaseModule implements OnBeforeSetUpListener, OnAfterTearDownListener {

  public function onBeforeSetUp(TestCase $testCase, $testName, $args) {
    // ...
  }

  public function onAfterTearDown(TestCase $testCase, $testName, $args) {
    // ...
  }

}

?>

(maybe not split it by each event separate but for example to SetUpTearDownListener, RunListener, TestResultListenter). New Listeners for new events then can be added in future.

Extended interface (with additional events added in future)

<?php

interface IExtendedTestCaseListener extends ITestCaseListener {

  public function onNewEvent();

}

?>

I personally preffer first solution.

Last edited by mystik (2015-08-17 15:53)

mystik

Usage I imagined should look like:

<?php

class MyTestCase extends TestCase {

  private $db1;
  private $db2;
  private $mockery;
  private $container;

  public function  __construct() {
    $this->db1 = $this->addListener(new DatabaseModule())
      ->loadConnection(__DIR__."/app/config.neon")
      ->purgeOnSetUp()
      ->loadOnSetUp(__DIR__."/schema.sql");

    $this->db2 = $this->addListener(new DatabaseModule())
      ->setConnection("localhost", "user", "password", "dbname");

    $this->mockery = $this->addListener(new MockeryModule());

    $this->container = $this->addListener(new ContainerModule())
      ->addConfig(__DIR__."/app/config.neon");
  }

}

class MyTestSomething extends MyTestCase {

  public function __construct() {
    parent::__construct();
    $this->container->addConfig(__DIR__."/app/test.config.neon");
  }

  public function setUp() {
    $this->db2->load(__DIR__."testingData.sql");
  }

  public function testSomething() {
    $mock = $this->mockery->mock("Class");
    $mock->shouldReceive("something")->once();
    // ...
    $connection = $this->db1->getConnection();
    //...
    $this->db2->load(__DIR__."/data2.sql");
    //...
    $this->db2->assertCount(10, "SELECT * FROM table WHERE a=1");
  }

}
?>

Last edited by mystik (2015-08-17 16:12)

mystik

I imlepmented in in our own fork so we can start using it. See https://github.com/…ter/pull/249

Milo

@mystik Sorry it takes too long.

Personally, I'm against any interface (for now). The reason is simple. Once we define an interface, any method name/signature change is a big BC break. I'm not sure we can design an interfaces well on a first shot.

I tried to find some use case in my libraries where I could test such event system. And I didn't. Unfortunatelly I have no time to write some just for fun/testing purpose. It is a next reason why I don't want to code from top of my head.

So, that's why I proposed the solution with a callback. It's harder to use, but simplier to maintain.

Maybe @DavidGrudl @FilipProcházka or @enumag have more skills with event systems. Any practical experiencies share is welcomed.

enumag

@Milo I don't have much experience with Nette/Tester but you might want to check how it is solved in Codeception: https://github.com/…n/Subscriber. It's a bit of a compromise. Maybe you and @mystik will like it.

Also it's similar to Kdyby/Events which has even better API in my opinion:

class FooListener extends Nette\Object implements Kdyby\Events\Subscriber
{
    public function getSubscribedEvents()
    {
        return array('App\OrderProcess::onSuccess');
    }

    public function onSuccess(OrderProcess $process)
    {
        // this will get called on order process success
    }
}

You could use something very similar.

Last edited by enumag (2015-09-17 22:26)