FrancisChung

Testing with Phpunit, Processwire & PHPStorm

14 posts in this topic

I'm writing this to give back something to the community that has given so much up front over the past year.
I noticed there's hardly any discussion about testing in these forums so I decided to write this quick primer to get some discussion going.

I'm by no means an expert on phpunit or selenium but I had to jump through a few hoops to get it working (especially with PHPStorm), so I thought I figured I should share my experiences with the community. 

Also, I'm hoping non Phpstorm users can still  pick something up useful in this post.


Prerequisites : It is assumed Phpunit (https://phpunit.de/) is installed via Composer,   Selenium (http://www.seleniumhq.org/) and Php-webdriver for Selenium (https://github.com/facebook/php-webdriver) is preinstalled.

For Phpstorm users, there's a fairly detailed installation and unit testing instructions here (https://www.jetbrains.com/help/phpstorm/2016.1/testing.html)
I found some parts of it leaving me with unanswered questions, so I'm hoping this post will supplement any questions that you might encounter.

Rather than writing a single monolithic post, I will write several posts covering different topics.

3 people like this

Share this post


Link to post
Share on other sites

Yeah Testing is quite important, sadly i will admit, I haven't been testing lately, however I will watch out for this space as I heavily use PHPUnit before and it's quite important please feel free to post, we will contribute to this ;) 

Share this post


Link to post
Share on other sites

PHPUnit configuration in PHP Storm :
This is where I got stuck for a while. The documentation surrounding this was unclear or unspecific at best.

The trick was to configure PHPStorm (in preferences) to use a Custom Autoloader that you specify and also define a default configuration file, both which were not apparent to me with the available documentation I could find.

The Custom Autoloader needs to point to autoload.php for Composer (hence my earlier pre-requisite of installing Phpunit via Composer).

The default configuration file is a XML file that defines certain basic variables that PHPUnit in order to function.

The one I've attached is a fairly basic one, with custom testsuites defined being the main difference.  The custom testsuites elements will become apparent once I explain how I structured my tests.  

For those interested in a detailed breakdown of the XML config, there are  some detailed documentation at  https://phpunit.de/manual/current/en/appendixes.configuration.html.

I've attached a screenshot of my preferences

PHPUnit PHPStorm config.png

phpunit.xml

1 person likes this

Share this post


Link to post
Share on other sites

Test Structure:

Now that we've setup PHPUnit with PHPStorm, let's talk about how to structure and setup your tests. This is where people can vary in their strategy and approaches quite a bit. Would be great to see how others approach it.

It's probably a good idea to isolate your tests away from your code. For example I've put my tests in /site/templates/php/tests

I've structured my tests into 3 basic folders that represent the different servers my website resides on.
They are core, uat and live.

The core folder contains the basic core versions of my tests. They also point to my local dev server for testing purposes.

The uat and live folder contains derived versions of these core tests that points to UAT & Live servers respectively.

The idea with having 3 seperate folders is that you can run isolated tests on each servers easily during testing and deployment phases.


Hopefully it's all been straight forward so far. I will go into detail about the tests themselves in the next post.






 

Test directory structure.png

1 person likes this

Share this post


Link to post
Share on other sites

Anatomy of a Test (Part 1):

Now that we have a test structure in place, you might be asking yourself "what sort of tests should I be writing?"

In general, you should start writing tests that would test the most important aspect of your site / system. 

For a CMS like Processwire, that would be tests that would cover Content Integrity for example, as content is king in a CMS.
Tests that would cover navigation could be another example as growing content means growing navigational possibilities.

You can use tools to determine how much of your written unit tests actually test your site/system, but test coverage is beyond the scope of this post.

Let's look at example content unit test to see what's actually inside.
 

<?php
/**
 * Created by PhpStorm.
 * User: FrancisChung
 * Date: 09/03/16
 * Time: 19:06
 */


namespace Test;
use Facebook\WebDriver\Remote\DesiredCapabilities;
use Facebook\WebDriver\Remote\RemoteWebDriver;
//use Facebook\WebDriver\WebDriverCapabilities;

require_once(__DIR__."/../../../composer/vendor/autoload.php");

class WebFingerspieleContentTest extends \PHPUnit_Framework_TestCase {
    protected $site;

    protected function setUp()
    {
        $this->site="http://localhost";
    }

    public function testSafari()
    {
        $browser = RemoteWebDriver::create('http://localhost:4444/wd/hub', DesiredCapabilities::safari());
        $this::TestPageTitle($browser);
        $browser->close();
        $browser->quit();
        $browser = null;

    }


    public function testFirefox() {
    $browser = RemoteWebDriver::create('http://localhost:4444/wd/hub', DesiredCapabilities::firefox());
    $this::TestPageTitle($browser);
    $browser->close();
    $browser = null;
    }


    public function testChrome() {
    $browser = RemoteWebDriver::create('http://localhost:4444/wd/hub', DesiredCapabilities::chrome());
    $this::TestPageTitle($browser);
    $browser->close();
    $browser = null;
    }


    private function TestData()
    {
        return array(
            array("{$this->site}/fingerspiele/alle-fingerspiele/10-kleine-zappelmaenner/", 'Zehn kleine Zappelmänner | Fingerspiel | Sprachspielspass.de'),
            array("{$this->site}/fingerspiele/alle-fingerspiele/der-osterhase/", 'Der Osterhase | Fingerspiel für Kinder | Sprachspielspass.de'),
            array("{$this->site}/fingerspiele/alle-fingerspiele/weiss-wie-schnee/",'Weiß wie Schnee | Fingerspiel Sommer | Sprachspielspass.de'),
            array("{$this->site}/fingerspiele/alle-fingerspiele/ein-kleines-auto/", 'Ein kleines Auto | Fingerspiel | Sprachspielspass.de'),
            array("{$this->site}/fingerspiele/alle-fingerspiele/es-regnet-ganz-sacht/", 'Es regnet ganz sacht | Fingerspiel | Sprachspielspass.de'),
            array("{$this->site}/fingerspiele/alle-fingerspiele/das-ist-der-daumen/", 'Das ist der Daumen | Fingerspiel | Sprachspielspass.de'),
            array("{$this->site}/fingerspiele/alle-fingerspiele/plumps-und-platsch/", "Plumps und platsch | Fingerspiel | Sprachspielspass.de"),
            array("{$this->site}/fingerspiele/alle-fingerspiele/fuenf-engelchen/", 'Fünf Engelchen | Fingerspiel Winter | Sprachspielspass.de'),
            array("{$this->site}/fingerspiele/alle-fingerspiele/erst-kommt-die-schnecke/", 'Erst kommt die Schnecke | Fingerspiel | Sprachspielspass.de')
        );
        //

    }


    /**
     * @param $browser
     */
    private function TestPageTitle($browser)
    {
        $data = $this::TestData();
        //TestFactory::PageTitleTest($data, $browser, $this);

        foreach($data as $testItem) {
            $url = $testItem[0];
            $expected = $testItem[1];
            $browser->get($url);
            $this->assertContains($expected, $browser->getTitle());
        }
    }
}


Now the 2 use statements  (DesiredCapabilities & Webdriver) are needed to access the Php-webdriver that we will be using to interact with Selenium. 

The require_once(__DIR__."/../../../composer/vendor/autoload.php") statement is to reference the Phpunit test framework that we installed via Composer (I got stuck on here for a while).

Your test also needs to derive from \PHPUnit_Framework_TestCase as you can see in the above example.

This is all the dependencies you need to write a Phpunit test for Processwire that runs on Phpstorm.

In the next post (Part 2 of this topic), I will go through each method and explain their purpose and function.

2 people like this

Share this post


Link to post
Share on other sites

@FrancisChung Thanks for writing this tutorial, but I'd suggest changing the title. You're describing functional or integration tests and not unit tests. Unit tests try to test the smallest units of software with as few external dependencies as possible (e.g. a method or a class). Testing browser output is possibly the largest testable unit of a website. I know that these names are almost never used 100% correctly, but I think in this case it would really make sense to correct the terminology.

1 person likes this

Share this post


Link to post
Share on other sites
50 minutes ago, LostKobrakai said:

@FrancisChung Thanks for writing this tutorial, but I'd suggest changing the title. You're describing functional or integration tests and not unit tests. Unit tests try to test the smallest units of software with as few external dependencies as possible (e.g. a method or a class). Testing browser output is possibly the largest testable unit of a website. I know that these names are almost never used 100% correctly, but I think in this case it would really make sense to correct the terminology.

 

Understand where you're coming from. Unfortunately, a lot of my classes actually have a simple output method and that method would output what is seen in the browser in its entirety so the smallest testable unit for most of my classes is also the largest. I've changed or removed terminology to remove any ambiguity or confusion.
 

Share this post


Link to post
Share on other sites

Anatomy of a Test (Part 2):

Now let's dive in a little and examine each of the functions to give you a better idea of what's going on.

    protected function setUp()
    {
        $this->site="http://localhost";
    }

setUp:

This function is called by Phpunit (note the lowercase Camel spelling, as it will break if not spelt like this) to setup your test fixtures. In a nutshell, this is the place to setup any dependencies for the tests. This is a function you should implement for each Test class.


I use this to mainly specify the URL of the site I'm going to test. As you will see later, I can easily reuse this test on other sites using this feature.

More info on fixtures can be found here --> https://phpunit.de/manual/current/en/fixtures.html

 

    public function testSafari()
    {
        $browser = RemoteWebDriver::create('http://localhost:4444/wd/hub', DesiredCapabilities::safari());
        $this::TestPageTitle($browser);
        $browser->close();
        $browser->quit();
        $browser = null;

    }


    public function testFirefox() {
        $browser = RemoteWebDriver::create('http://localhost:4444/wd/hub', DesiredCapabilities::firefox());
        $this::TestPageTitle($browser);
        $browser->close();
        $browser = null;
    }


    public function testChrome() {
        $browser = RemoteWebDriver::create('http://localhost:4444/wd/hub', DesiredCapabilities::chrome());
        $this::TestPageTitle($browser);
        $browser->close();
        $browser = null;
    }

test<Browser>:

Any functions with a "test" prefix (note lowercase spelling here) will be called by the Phpunit framework, so you can see I have 3 test functions for the framework to call.

Each of my "external" test functions represent a test case corresponding to a particular browser. 
Obviously you don't have to set it up like this, but my choices were somewhat dictated by my dependency on Php-webdriver.

The first line of code is invoking the Php-webdriver and passing it the location of the <xxx> and telling it what properties (DesiredCapabilities) you want.
Once this is successful, I call some internal test functions which I will go through next.

    private function TestData()
    {
        return array(
            array("{$this->site}/fingerspiele/alle-fingerspiele/10-kleine-zappelmaenner/", 'Zehn kleine Zappelmänner | Fingerspiel | Sprachspielspass.de'),
            array("{$this->site}/fingerspiele/alle-fingerspiele/der-osterhase/", 'Der Osterhase | Fingerspiel für Kinder | Sprachspielspass.de'),
            array("{$this->site}/fingerspiele/alle-fingerspiele/weiss-wie-schnee/",'Weiß wie Schnee | Fingerspiel Sommer | Sprachspielspass.de'),
            array("{$this->site}/fingerspiele/alle-fingerspiele/ein-kleines-auto/", 'Ein kleines Auto | Fingerspiel | Sprachspielspass.de'),
            array("{$this->site}/fingerspiele/alle-fingerspiele/es-regnet-ganz-sacht/", 'Es regnet ganz sacht | Fingerspiel | Sprachspielspass.de'),
            array("{$this->site}/fingerspiele/alle-fingerspiele/das-ist-der-daumen/", 'Das ist der Daumen | Fingerspiel | Sprachspielspass.de'),
            array("{$this->site}/fingerspiele/alle-fingerspiele/plumps-und-platsch/", "Plumps und platsch | Fingerspiel | Sprachspielspass.de"),
            array("{$this->site}/fingerspiele/alle-fingerspiele/fuenf-engelchen/", 'Fünf Engelchen | Fingerspiel Winter | Sprachspielspass.de'),
            array("{$this->site}/fingerspiele/alle-fingerspiele/erst-kommt-die-schnecke/", 'Erst kommt die Schnecke | Fingerspiel | Sprachspielspass.de')
        );
        //

    }

    private function TestPageTitle($browser)
    {
        $data = $this::TestData();
        //TestFactory::PageTitleTest($data, $browser, $this);

        foreach($data as $testItem) {
            $url = $testItem[0];
            $expected = $testItem[1];
            $browser->get($url);
            $this->assertContains($expected, $browser->getTitle());
        }
    }

Test<TestName>:

These functions should represent each unit of testing you're interested in. So in my example, I'm testing to see a web page has to expected page title.

So TestPageTitle will loop through a set of test URLs and expected titles and report any failures using the assertContains.

 $this->assertContains($expected, $browser->getTitle());

By all means, there are other ways to assert your tests other than assertContains, so feel free to explore them.

Edited by FrancisChung
More info on setUp method

Share this post


Link to post
Share on other sites

Test Leveraging:

One of the advantages of setting up your tests like this is that you can easily create tests for other sites by merely creating a subclass and defining the URL of the new site you want to test.

class WebFingerspieleContentUATest extends WebFingerspieleContentTest {

    protected function setUp()
    {
        $this->site = "http://finger-spiele.com";
    }


    protected function tearDown()
    {

    }
}

Also note that in my earlier post, Test Structure, I've organised the tests separately per site. So it's easy to run tests or a set of tests on one or multiple sites. 
It certainly saved me a great deal of time testing during UAT, Live deployment .... 

 

Final Notes:

1) The way I've structured is by no means the best or even right(?) way for you. I would like to hear suggestions or alternative ways of structuring your tests.

2) The Selenium test framework is not always up to date with the latest browser updates, so there's a chance that your browser that you're testing does not launch.
As time of writing, I'm aware of a problem with Firefox and maybe Chrome as well. It certainly wasn't the case when I was using it extensively.
Unfortunately, I don't have a good workaround of this except perhaps rolling back your browser to a last known working version.

3) I should add that you need to have the Selenium server running in the background for the tests to work. 
 

java -jar selenium-server-standalone-2.50.1.jar

4) For chrome, I think you need a chromewebdriver in your Selenium folder and execute the following

java -Dwebdriver.chrome.driver="/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome" -jar selenium-server-standalone-2.53.0.jar

5) This concludes the mini tutorial. I hope it was informative to some one out there, and please leave some comments. Would love some feedback. 

Share this post


Link to post
Share on other sites

quite an old thread, but I started to test some basic functionality and I can't get this to work... :(

I don't use selenium but only PHPUnit to test some functions within modules. But whatever I try it fails.

Basically I can decide whether I want to see this message:

PDOException: You cannot serialize or unserialize PDO instances

or - if I follow these rules https://blogs.kent.ac.uk/webdev/2011/07/14/phpunit-and-unserialized-pdo-instances/ -

ERROR: application encountered an error and can not continue. Error was logged.

 

Anyone who struggled with this as well? And - more important - any suggestions or solutions?

 

big thanks!

 

btw @FrancisChung: based on the stackoverflow post I think you just pasted the annotations at the wrong place. It must be outside the class {} ;) 

Share this post


Link to post
Share on other sites
6 hours ago, chrizz said:

 

btw @FrancisChung: based on the stackoverflow post I think you just pasted the annotations at the wrong place. It must be outside the class {} ;) 

2

Which annotations are you referring to, and which class {} ?

Share this post


Link to post
Share on other sites

i was referring to your post here:

https://stackoverflow.com/questions/34225720/keep-getting-you-cannot-serialize-or-unserialize-pdo-instances-in-phpunit-usin

namespace Test;

include_once(__DIR__."/../../../../index.php");     //Bootstrap to Processwire CMS

class ImageTest extends \PHPUnit_Framework_TestCase {

    /**
     * @backupGlobals disabled
     * @backupStaticAttributes disabled
     * @runTestsInSeparateProcesses
     * @runInSeparateProcess
     * @preserveGlobalState disabled
     */


    protected function setUp()
    {
        //$this->testpages = wire(pages)->find("template=fingergames|template=songs|template=poems");
    }

[...]
}

 

And no, I am not using PHPStorm but Netbeans. Finally I managed it use PHPUnit with PW modules. 

In the end it was a combination of a config XML

backupGlobals="false"
backupStaticAttributes="false"

bootstrapping the PW index.php

and include a setUp method within the test class with the correct namespaces

Example (maybe someone needs something like this)

<?php  
use PHPUnit\Framework\TestCase;

class testMe extends TestCase {
	
  protected function setUp() {
	global $wire;
	$this->wire = $wire;
  }
	
  public function test_getNextRuntimeFromCycle() {
	$res = $this->wire->modules->get("testMe")->myMethodWorks();
	$this->assertEquals(true, $res);
  }	
}

 

 

 

Share this post


Link to post
Share on other sites
    /**
     * @backupGlobals disabled
     * @backupStaticAttributes disabled
     * @runTestsInSeparateProcesses
     * @runInSeparateProcess
     * @preserveGlobalState disabled
     */

When you say annotations, if you are referring to the above, then in my project they are inside the class definition.
If you managed to get it working outside the class, then perhaps it doesn't matter as much whether they stay inside or outside the class.


If the PHPUnit documentation says it should reside outside the class, then people should have that in mind if they are still getting errors.

I've reviewed my tutorial and whilst I did describe the setUp method in detail, I probably should mention that it's a method you must include and implement. I will change it accordingly. 

 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By Richard Jedlička
      Tense    
      Tense (Test ENvironment Setup & Execution) is a command-line tool to easily run tests agains multiple versions of ProcessWire CMF.
      Are you building a module, or a template and you need to make sure it works in all supported ProcessWire versions? Then Tense is exactly what you need. Write the tests in any testing framework, tell Tense which ProcessWire versions you are interested in and it will do the rest for you.

      See example or see usage in a real project.
      How to use?
      1. Install it: 
      composer global require uiii/tense 2. Create tense.yml config:
      tense init 3. Run it:
      tense run  
      For detailed instructions see Github page: https://github.com/uiii/tense
       
      This is made possible thanks to the great wireshell tool by @justb3a, @marcus and others.
       
      What do you think about it? Do you find it useful? Do you have some idea? Did you find some bug? Tell me you opinion. Write it here or in the issue tracker.
    • By hettiger
      I'm using GitLab CI for continuous integration right now. Other people might use something like Travis CI. For the frontend part things are pretty easy:
      image: node:6.10.3 cache: paths: - node_modules/ build: script: - npm install - node_modules/.bin/gulp build Now I could extend my setup to run tests or whatever I want. It just works.
      In this post I'm not interested into finding out how I could go about testing and stuff. I just want to know how I could accomplish the equivalent to the above setup task for ProcessWire. It doesn't seem to be too easy to me since the installation process is running in the browser asking you a lot of stuff. How could I go about it using the CLI?
      That's what I currently have:
      git clone git@github.com:processwire/processwire.git cd processwire mv site-blank site rm .gitignore git submodule add -b develop git@gitlab.local:path/to/MyModule.git site/modules/MyModule From here on I'm stuck.
      The only possible solution I see is maintaining a super repository including a database dump that has it's module dependencies defined in a .gitmodules file. This could then be installed using:
      git clone --recursive git@... Once that's completed all that needs to be done would be to create the database using the dump.
      Probably that's a pretty solid solution but it adds the overhead of having to maintain an additional repository +  database dump. I'm trying to find an alternative that's using the original processwire repository. That way I could just rerun my build once a new version get's merged into master and see if everything would be still working as well.
      I'm looking forward to your replies. Thanks!
    • By FrancisChung
      Hi fellow Processwirers (?),
      I was wondering if anyone here uses PHPStorm in conjunction with PhpUnit?
      In particular, I'm trying to get it to work with Processwire so I can incorporate the Pages & other intrinsic Processwire objects into the testing.
      Any example or real life configuration examples would be most welcome.

      Thanks in advance.
    • By FrancisChung
      Hi there,
      I'm a newbie and I'm trying to learn more about Processwire so I've setup XDebug + PHPStorm to get some real time debugging happening. 

      I've noticed that no matter where i put a breakpoint in any of my php modules, it seems to return the following error.
      Error: __debuginfo() must return an array (line 55 of /Users/FrancisChung/site/Test/templates/head.inc) 
      This error message was shown because you are logged in as a Superuser. Error has been logged.
      I was wondering if fellow Processwire users have a similar setup, and was successful in getting debugging going.
      I've also read this forum post:
      https://processwire.com/talk/topic/1611-yes-debugging-templates-and-core-code-works/
      I'm also aware there's an issue with XDebug and Processwire:
      https://github.com/ryancramerdesign/ProcessWire/issues/1316