Testing with Phpunit, Processwire & PHPStorm

[FrancisChung]

I'm writing this to give back something to the community that has given so much up front over the past year.
I noticed there's hardly any discussion about testing in these forums so I decided to write this quick primer to get some discussion going.

I'm by no means an expert on phpunit or selenium but I had to jump through a few hoops to get it working (especially with PHPStorm), so I thought I figured I should share my experiences with the community. 

Also, I'm hoping non Phpstorm users can still  pick something up useful in this post.

Prerequisites : It is assumed Phpunit (https://phpunit.de/) is installed via Composer,   Selenium (http://www.seleniumhq.org/) and Php-webdriver for Selenium (https://github.com/facebook/php-webdriver) is preinstalled.

For Phpstorm users, there's a fairly detailed installation and unit testing instructions here (https://www.jetbrains.com/help/phpstorm/2016.1/testing.html)
I found some parts of it leaving me with unanswered questions, so I'm hoping this post will supplement any questions that you might encounter.

Rather than writing a single monolithic post, I will write several posts covering different topics.

[qtguru]

Yeah Testing is quite important, sadly i will admit, I haven't been testing lately, however I will watch out for this space as I heavily use PHPUnit before and it's quite important please feel free to post, we will contribute to this  

[FrancisChung]

PHPUnit configuration in PHP Storm :
This is where I got stuck for a while. The documentation surrounding this was unclear or unspecific at best.

The trick was to configure PHPStorm (in preferences) to use a Custom Autoloader that you specify and also define a default configuration file, both which were not apparent to me with the available documentation I could find.

The Custom Autoloader needs to point to autoload.php for Composer (hence my earlier pre-requisite of installing Phpunit via Composer).

The default configuration file is a XML file that defines certain basic variables that PHPUnit in order to function.

The one I've attached is a fairly basic one, with custom testsuites defined being the main difference.  The custom testsuites elements will become apparent once I explain how I structured my tests.  

For those interested in a detailed breakdown of the XML config, there are  some detailed documentation at  https://phpunit.de/manual/current/en/appendixes.configuration.html.

I've attached a screenshot of my preferences

phpunit.xml

[FrancisChung]

Test Structure:

Now that we've setup PHPUnit with PHPStorm, let's talk about how to structure and setup your tests. This is where people can vary in their strategy and approaches quite a bit. Would be great to see how others approach it.

It's probably a good idea to isolate your tests away from your code. For example I've put my tests in /site/templates/php/tests

I've structured my tests into 3 basic folders that represent the different servers my website resides on.
They are core, uat and live.

The core folder contains the basic core versions of my tests. They also point to my local dev server for testing purposes.

The uat and live folder contains derived versions of these core tests that points to UAT & Live servers respectively.

The idea with having 3 seperate folders is that you can run isolated tests on each servers easily during testing and deployment phases.

Hopefully it's all been straight forward so far. I will go into detail about the tests themselves in the next post.

 

[FrancisChung]

Anatomy of a Test (Part 1):

Now that we have a test structure in place, you might be asking yourself "what sort of tests should I be writing?"

In general, you should start writing tests that would test the most important aspect of your site / system. 

For a CMS like Processwire, that would be tests that would cover Content Integrity for example, as content is king in a CMS.
Tests that would cover navigation could be another example as growing content means growing navigational possibilities.

You can use tools to determine how much of your written unit tests actually test your site/system, but test coverage is beyond the scope of this post.

Let's look at example content unit test to see what's actually inside.
 

<?php
/**
 * Created by PhpStorm.
 * User: FrancisChung
 * Date: 09/03/16
 * Time: 19:06
 */


namespace Test;
use Facebook\WebDriver\Remote\DesiredCapabilities;
use Facebook\WebDriver\Remote\RemoteWebDriver;
//use Facebook\WebDriver\WebDriverCapabilities;

require_once(__DIR__."/../../../composer/vendor/autoload.php");

class WebFingerspieleContentTest extends \PHPUnit_Framework_TestCase {
    protected $site;

    protected function setUp()
    {
        $this->site="http://localhost";
    }

    public function testSafari()
    {
        $browser = RemoteWebDriver::create('http://localhost:4444/wd/hub', DesiredCapabilities::safari());
        $this::TestPageTitle($browser);
        $browser->close();
        $browser->quit();
        $browser = null;

    }


    public function testFirefox() {
    $browser = RemoteWebDriver::create('http://localhost:4444/wd/hub', DesiredCapabilities::firefox());
    $this::TestPageTitle($browser);
    $browser->close();
    $browser = null;
    }


    public function testChrome() {
    $browser = RemoteWebDriver::create('http://localhost:4444/wd/hub', DesiredCapabilities::chrome());
    $this::TestPageTitle($browser);
    $browser->close();
    $browser = null;
    }


    private function TestData()
    {
        return array(
            array("{$this->site}/fingerspiele/alle-fingerspiele/10-kleine-zappelmaenner/", 'Zehn kleine Zappelmänner | Fingerspiel | Sprachspielspass.de'),
            array("{$this->site}/fingerspiele/alle-fingerspiele/der-osterhase/", 'Der Osterhase | Fingerspiel für Kinder | Sprachspielspass.de'),
            array("{$this->site}/fingerspiele/alle-fingerspiele/weiss-wie-schnee/",'Weiß wie Schnee | Fingerspiel Sommer | Sprachspielspass.de'),
            array("{$this->site}/fingerspiele/alle-fingerspiele/ein-kleines-auto/", 'Ein kleines Auto | Fingerspiel | Sprachspielspass.de'),
            array("{$this->site}/fingerspiele/alle-fingerspiele/es-regnet-ganz-sacht/", 'Es regnet ganz sacht | Fingerspiel | Sprachspielspass.de'),
            array("{$this->site}/fingerspiele/alle-fingerspiele/das-ist-der-daumen/", 'Das ist der Daumen | Fingerspiel | Sprachspielspass.de'),
            array("{$this->site}/fingerspiele/alle-fingerspiele/plumps-und-platsch/", "Plumps und platsch | Fingerspiel | Sprachspielspass.de"),
            array("{$this->site}/fingerspiele/alle-fingerspiele/fuenf-engelchen/", 'Fünf Engelchen | Fingerspiel Winter | Sprachspielspass.de'),
            array("{$this->site}/fingerspiele/alle-fingerspiele/erst-kommt-die-schnecke/", 'Erst kommt die Schnecke | Fingerspiel | Sprachspielspass.de')
        );
        //

    }


    /**
     * @param $browser
     */
    private function TestPageTitle($browser)
    {
        $data = $this::TestData();
        //TestFactory::PageTitleTest($data, $browser, $this);

        foreach($data as $testItem) {
            $url = $testItem[0];
            $expected = $testItem[1];
            $browser->get($url);
            $this->assertContains($expected, $browser->getTitle());
        }
    }
}

Now the 2 use statements  (DesiredCapabilities & Webdriver) are needed to access the Php-webdriver that we will be using to interact with Selenium. 

The require_once(__DIR__."/../../../composer/vendor/autoload.php") statement is to reference the Phpunit test framework that we installed via Composer (I got stuck on here for a while).

Your test also needs to derive from \PHPUnit_Framework_TestCase as you can see in the above example.

This is all the dependencies you need to write a Phpunit test for Processwire that runs on Phpstorm.

In the next post (Part 2 of this topic), I will go through each method and explain their purpose and function.

[LostKobrakai]

@FrancisChung Thanks for writing this tutorial, but I'd suggest changing the title. You're describing functional or integration tests and not unit tests. Unit tests try to test the smallest units of software with as few external dependencies as possible (e.g. a method or a class). Testing browser output is possibly the largest testable unit of a website. I know that these names are almost never used 100% correctly, but I think in this case it would really make sense to correct the terminology.

[FrancisChung]

50 minutes ago, LostKobrakai said:

@FrancisChung Thanks for writing this tutorial, but I'd suggest changing the title. You're describing functional or integration tests and not unit tests. Unit tests try to test the smallest units of software with as few external dependencies as possible (e.g. a method or a class). Testing browser output is possibly the largest testable unit of a website. I know that these names are almost never used 100% correctly, but I think in this case it would really make sense to correct the terminology.

 

Understand where you're coming from. Unfortunately, a lot of my classes actually have a simple output method and that method would output what is seen in the browser in its entirety so the smallest testable unit for most of my classes is also the largest. I've changed or removed terminology to remove any ambiguity or confusion.
 

[FrancisChung]

Anatomy of a Test (Part 2):

Now let's dive in a little and examine each of the functions to give you a better idea of what's going on.

    protected function setUp()
    {
        $this->site="http://localhost";
    }

setUp:

This function is called by Phpunit (note the lowercase Camel spelling, as it will break if not spelt like this) to setup your test fixtures. In a nutshell, this is the place to setup any dependencies for the tests. This is a function you should implement for each Test class.

I use this to mainly specify the URL of the site I'm going to test. As you will see later, I can easily reuse this test on other sites using this feature.

More info on fixtures can be found here --> https://phpunit.de/manual/current/en/fixtures.html

 

    public function testSafari()
    {
        $browser = RemoteWebDriver::create('http://localhost:4444/wd/hub', DesiredCapabilities::safari());
        $this::TestPageTitle($browser);
        $browser->close();
        $browser->quit();
        $browser = null;

    }


    public function testFirefox() {
        $browser = RemoteWebDriver::create('http://localhost:4444/wd/hub', DesiredCapabilities::firefox());
        $this::TestPageTitle($browser);
        $browser->close();
        $browser = null;
    }


    public function testChrome() {
        $browser = RemoteWebDriver::create('http://localhost:4444/wd/hub', DesiredCapabilities::chrome());
        $this::TestPageTitle($browser);
        $browser->close();
        $browser = null;
    }

test<Browser>:

Any functions with a "test" prefix (note lowercase spelling here) will be called by the Phpunit framework, so you can see I have 3 test functions for the framework to call.

Each of my "external" test functions represent a test case corresponding to a particular browser. 
Obviously you don't have to set it up like this, but my choices were somewhat dictated by my dependency on Php-webdriver.

The first line of code is invoking the Php-webdriver and passing it the location of the <xxx> and telling it what properties (DesiredCapabilities) you want.
Once this is successful, I call some internal test functions which I will go through next.

    private function TestData()
    {
        return array(
            array("{$this->site}/fingerspiele/alle-fingerspiele/10-kleine-zappelmaenner/", 'Zehn kleine Zappelmänner | Fingerspiel | Sprachspielspass.de'),
            array("{$this->site}/fingerspiele/alle-fingerspiele/der-osterhase/", 'Der Osterhase | Fingerspiel für Kinder | Sprachspielspass.de'),
            array("{$this->site}/fingerspiele/alle-fingerspiele/weiss-wie-schnee/",'Weiß wie Schnee | Fingerspiel Sommer | Sprachspielspass.de'),
            array("{$this->site}/fingerspiele/alle-fingerspiele/ein-kleines-auto/", 'Ein kleines Auto | Fingerspiel | Sprachspielspass.de'),
            array("{$this->site}/fingerspiele/alle-fingerspiele/es-regnet-ganz-sacht/", 'Es regnet ganz sacht | Fingerspiel | Sprachspielspass.de'),
            array("{$this->site}/fingerspiele/alle-fingerspiele/das-ist-der-daumen/", 'Das ist der Daumen | Fingerspiel | Sprachspielspass.de'),
            array("{$this->site}/fingerspiele/alle-fingerspiele/plumps-und-platsch/", "Plumps und platsch | Fingerspiel | Sprachspielspass.de"),
            array("{$this->site}/fingerspiele/alle-fingerspiele/fuenf-engelchen/", 'Fünf Engelchen | Fingerspiel Winter | Sprachspielspass.de'),
            array("{$this->site}/fingerspiele/alle-fingerspiele/erst-kommt-die-schnecke/", 'Erst kommt die Schnecke | Fingerspiel | Sprachspielspass.de')
        );
        //

    }

    private function TestPageTitle($browser)
    {
        $data = $this::TestData();
        //TestFactory::PageTitleTest($data, $browser, $this);

        foreach($data as $testItem) {
            $url = $testItem[0];
            $expected = $testItem[1];
            $browser->get($url);
            $this->assertContains($expected, $browser->getTitle());
        }
    }

Test<TestName>:

These functions should represent each unit of testing you're interested in. So in my example, I'm testing to see a web page has to expected page title.

So TestPageTitle will loop through a set of test URLs and expected titles and report any failures using the assertContains.

 $this->assertContains($expected, $browser->getTitle());

By all means, there are other ways to assert your tests other than assertContains, so feel free to explore them.

Edited July 9, 2017 by FrancisChung
More info on setUp method

[FrancisChung]

Test Leveraging:

One of the advantages of setting up your tests like this is that you can easily create tests for other sites by merely creating a subclass and defining the URL of the new site you want to test.

class WebFingerspieleContentUATest extends WebFingerspieleContentTest {

    protected function setUp()
    {
        $this->site = "http://finger-spiele.com";
    }


    protected function tearDown()
    {

    }
}

Also note that in my earlier post, Test Structure, I've organised the tests separately per site. So it's easy to run tests or a set of tests on one or multiple sites. 
It certainly saved me a great deal of time testing during UAT, Live deployment .... 

 

Final Notes:

1) The way I've structured is by no means the best or even right(?) way for you. I would like to hear suggestions or alternative ways of structuring your tests.

2) The Selenium test framework is not always up to date with the latest browser updates, so there's a chance that your browser that you're testing does not launch.
As time of writing, I'm aware of a problem with Firefox and maybe Chrome as well. It certainly wasn't the case when I was using it extensively.
Unfortunately, I don't have a good workaround of this except perhaps rolling back your browser to a last known working version.

3) I should add that you need to have the Selenium server running in the background for the tests to work. 
 

java -jar selenium-server-standalone-2.50.1.jar

4) For chrome, I think you need a chromewebdriver in your Selenium folder and execute the following

java -Dwebdriver.chrome.driver="/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome" -jar selenium-server-standalone-2.53.0.jar

5) This concludes the mini tutorial. I hope it was informative to some one out there, and please leave some comments. Would love some feedback. 

[chrizz]

quite an old thread, but I started to test some basic functionality and I can't get this to work...

I don't use selenium but only PHPUnit to test some functions within modules. But whatever I try it fails.

Basically I can decide whether I want to see this message:

PDOException: You cannot serialize or unserialize PDO instances

or - if I follow these rules https://blogs.kent.ac.uk/webdev/2011/07/14/phpunit-and-unserialized-pdo-instances/ -

ERROR: application encountered an error and can not continue. Error was logged.

 

Anyone who struggled with this as well? And - more important - any suggestions or solutions?

 

big thanks!

 

btw @FrancisChung: based on the stackoverflow post I think you just pasted the annotations at the wrong place. It must be outside the class {}  

[FrancisChung]

@chrizz,

Are you using PHPStorm?

 

[FrancisChung]

6 hours ago, chrizz said:

 

btw @FrancisChung: based on the stackoverflow post I think you just pasted the annotations at the wrong place. It must be outside the class {}  

2

Which annotations are you referring to, and which class {} ?

[chrizz]

i was referring to your post here:

https://stackoverflow.com/questions/34225720/keep-getting-you-cannot-serialize-or-unserialize-pdo-instances-in-phpunit-usin

namespace Test;

include_once(__DIR__."/../../../../index.php");     //Bootstrap to Processwire CMS

class ImageTest extends \PHPUnit_Framework_TestCase {

    /**
     * @backupGlobals disabled
     * @backupStaticAttributes disabled
     * @runTestsInSeparateProcesses
     * @runInSeparateProcess
     * @preserveGlobalState disabled
     */


    protected function setUp()
    {
        //$this->testpages = wire(pages)->find("template=fingergames|template=songs|template=poems");
    }

[...]
}

 

And no, I am not using PHPStorm but Netbeans. Finally I managed it use PHPUnit with PW modules. 

In the end it was a combination of a config XML

backupGlobals="false"
backupStaticAttributes="false"

bootstrapping the PW index.php

and include a setUp method within the test class with the correct namespaces

Example (maybe someone needs something like this)

<?php  
use PHPUnit\Framework\TestCase;

class testMe extends TestCase {
	
  protected function setUp() {
	global $wire;
	$this->wire = $wire;
  }
	
  public function test_getNextRuntimeFromCycle() {
	$res = $this->wire->modules->get("testMe")->myMethodWorks();
	$this->assertEquals(true, $res);
  }	
}

 

 

 

[FrancisChung]

    /**
     * @backupGlobals disabled
     * @backupStaticAttributes disabled
     * @runTestsInSeparateProcesses
     * @runInSeparateProcess
     * @preserveGlobalState disabled
     */

When you say annotations, if you are referring to the above, then in my project they are inside the class definition.
If you managed to get it working outside the class, then perhaps it doesn't matter as much whether they stay inside or outside the class.

If the PHPUnit documentation says it should reside outside the class, then people should have that in mind if they are still getting errors.

I've reviewed my tutorial and whilst I did describe the setUp method in detail, I probably should mention that it's a method you must include and implement. I will change it accordingly. 

 

[FrancisChung]

Database Mocking  :