How to setup Composer and use external libraries in ProcessWire

[MoritzLost]

I've been working with ProcessWire for a while now, and I've noticed that using Composer to manage dependencies and autoload external libraries isn't as prevalent in ProcessWire development as in other areas of PHP programming. I started out by using the default setup recommend in this blogpost. However, one major problem I have with this approach is that all external dependencies live in the webroot (the directory the server points to), which is unfavourable from a security standpoint and, in my opinion, just feels a bit messy.

In this tutorial, I want to go through a quick setup of Composer and ProcessWire that keeps the dependencies, all custom-written code and other source material outside of the webroot, and makes full usage of the Composer autoloader. This setup is pretty basic, so this tutorial is probably more useful to beginners (this is why I'll also include some general information on Composer), but hopefully everyone can take something away from this for their personal workflow.

Site structure after setup

This is what the directory structure can look like after the setup:

.
├── composer.json
├── composer.lock
├── node_modules
│   └── ...
├── public
│   ├── index.php
│   ├── site
│   ├── wire
│   └── ...
├── packacke-lock.json
├── package.json
├── sass
│   ├── main.scss
│   ├── _variables.scss
│   └── ...
├── src
│   ├── ContentBag.php
│   └── ...
└── vendor
    ├── autoload.php
    ├── composer
    ├── league
    ├── symfony
    └── ...

As mentioned, the main point of this setup is to keep all external libraries, all other custom source code and resources out of the webroot. That includes Composer's vendor folder, your node_modules and JavaScript source folder if you are compiling JavaScript with webpack or something similar and including external scripts via NPM, or your CSS preprocessor files if you are using SASS or LESS. In this setup, the public directory acts as the webroot (the directory that is used as the entry point by the server, DocumentRoot in the Apache configuration). So all other files and directories in the mysite folder aren't accessible over the web, even if something goes wrong.

One caveat of this setup is that it's not possible to install ProcessWire modules through Composer using the PW Module Installer (see Blogpost above), but that's just a minor inconvenience in my experience.

Installation

You'll need to have composer installed on your system for this. Installation guides can be found on getcomposer.org.

First, open up your shell and navigate to the mysite folder.

$ cd /path/to/mysite/

Now, we'll initialize a new Composer project:

$ composer init

The CLI will ask some questions about your projects. Some hints if you are unsure how to answer the prompts:

• Package names are in the format <vendor>/<project>, where vendor is your developer handle. I use my Github account, so I'll put moritzlost/mysite (all lowercase).
• Project type is project if you are creating a website.
• Author should be in the format Name <email>.
• Minimum Stability: I prefer stable, this way you only get stable versions of dependencies.
• License will be proprietary unless you plan on sharing your code under a FOSS license.
• Answer no to the interactive dependencies prompts.

This creates the composer.json file, which will be used to keep track of your dependencies. For now, you only need to run the composer install command to initialize the vendor directory and the autoloader:

$ composer install

Now it's time to download and install ProcessWire into the public directory:

$ git clone https://github.com/processwire/processwire public

If you don't use git, you can also download ProcessWire manually. I like to clean up the directory after that:

$ cd public
$ rm -r .git .gitattributes .gitignore CONTRIBUTING.md LICENSE.TXT README.md

Now, setup your development server to point to the /path/to/mysite/public/ directory (mind the public/ at the end!) and install ProcessWire normally.

Including & using the autoloader

With ProcessWire installed, we need to include the composer autoloader. If you check ProcessWire's index.php file, you'll see that it tries to include the autoloader if present. However, this assumes the vendor folder is inside the webroot, so it won't work in our case.

One good place to include the autoloader is using a site hook file. We need the autoloader as early as possible, so we'll use init.php:

EDIT: As @horst pointed out, it's much better to put this code inside the config.php file instead, as the autoloader will be included much earlier:

// public/site/config.php
<?php
namespace Processwire;

require '../../vendor/autoload.php';

The following also doesn't apply when including the autoloader in the config-file.

This has one caveat: Since this file is executed by ProcessWire after all modules had their init methods called, the autoloader will not be available in those. I haven't come across a case where I needed it this early so far; however, if you really need to include the autoloader earlier than that, you could just edit the lines in the index.php file linked above to include the correct autoloader path. In this case, make sure not to overwrite this when you update the core!

Now we can finally include external libraries and use them in our code without hassle! I'll give you an example. For one project, I needed to parse URLs and check some properties of the path, host et c. I could use parse_url, however that has a couple of downsides (specifically, it doesn't throw exceptions, but just fails silently). Since I didn't want to write a huge error-prone regex myself, I looked for a package that would help me out. I decided to use this URI parser, since it's included in the PHP League directory, which generally stands for high quality.

First, install the dependency (from the project root, the folder your composer.json file lives in):

$ composer require league/uri-parser

This will download the package into your vendor directory and refresh the autoloader.

Now you can just use the package in your own code, and composer will autoload the required class files:

// public/site/templates/basic-page.php
<?php
namespace Processwire;

use \League\Uri\Parser;

// ...
if ($url = $page->get('url')) {
	$parser = new Parser();
	$parsed_url = $parser->parse($url);
	// do stuff with $parsed_url ...
}

Wiring up custom classes and code

Another topic that I find really useful but often gets overlooked in Composer tutorials is the ability to wire up your own namespace to a folder. So if you want to write some object-oriented code outside of your template files, this gives you an easy way to autoload those using Composer as well. If you look at the tree above, you'll see there's a src/ directory inside the project root, and a ContentBag.php file inside. I want to connect classes in this directory with a custom namespace to be able to have them autoloaded when I use them in my templates.

To do this, you need to edit your composer.json file:

{
    "name": "moritzlost/mysite",
    "type": "project",
    "license": "proprietary",
    "authors": [
        {
            "name": "Moritz L'Hoest",
            "email": "info@herebedragons.world"
        }
    ],
    "minimum-stability": "stable",
    "require": {},
	"autoload": {
		"psr-4": {
			"MoritzLost\\MySite\\": "src/"
		}
	}
}

Most of this stuff was added during initialization, for now take note of the autoload information. The syntax is a bit tricky, since you have to escape the namespace seperator (backslash) with another backslash (see the documentation for more information). Also note the PSR-4 key, since that's the standard I use to namespace my classes.

The line "MoritzLost\\MySite\\": "src/" tells Composer to look for classes under the namespace \MoritzLost\MySite\ in the src/ directory in my project root. After adding the autoload information, you have to tell composer to refresh the autoloader information:

$ composer dump-autoload

Now I'm ready to use my classes in my templates. So, if I have this file:

// src/ContentBag.php
<?php
namespace MoritzLost\MySite;

class ContentBag {
	// class stuff
}

I can now use the ContentBag class freely in my templates without having to include those files manually:

// public/site/templates/home.php
<?php
namespace Processwire;

use MoritzLost\MySite\ContentBag;

$contentbag = new ContentBag();
// do stuff with contentbag ...

Awesome!

By the way, in PSR-4, sub-namespaces correspond to folders, so I can put the class MoritzLost\MySite\Stuff\SomeStuff in src/Stuff/SomeStuff.php and it will get autoloaded as well. If you have a lot of classes, you can group them this way.

Conclusion

With this setup, you are following secure practices and have much flexibility over what you want to include in your project. For example, you can just as well initialize a JavaScript project by typing npm init in the project root. You can also start tracking the source code of your project inside your src/ directory independently of the ProcessWire installation. All in all, you have good seperation of concerns between ProcessWire, external dependencies, your templates and your OOP-code, as well as another level of security should your Server or CGI-handler ever go AWOL. You can also build upon this approach. For example, it's good practice to keep credentials for your database outside the webroot. So you could modify the public/site/config.php file to include a config or .env file in your project root and read the database credentials from there.

Anyway, that's the setup I came up with. I'm sure it's not perfect yet; also this tutorial is probably missing some information or isn't detailed enough in some areas depending on your level of experience. Feel free to ask for clarification, and to point out the things I got wrong. I like to learn as well ?

Thanks for making it all the way to the bottom. Cheers!

[LostKobrakai]

Just to clarify the bit about "putting stuff outside the webroot" for security reasons: This is certainly a nice precaution, but processwire is not using it because there are way to many shared hosters, where it's simply not possible to go outside the webroot. I'd also say that if your webserver is not behaving correctly you've got bigger problems to deal with, so usually just adjusting your .htaccess file should be perfectly fine as well. Especially as your php process still needs to be able to read things even if they're not within the webroot to be able to execute them.

3 hours ago, MoritzLost said:

Another topic that I find really useful but often gets overlooked in Composer tutorials is the ability to wire up your own namespace to a folder.

There's not really a need for composer for that part as processwire has it's own psr-4 compatible classloader: https://processwire.com/api/ref/class-loader/

[MoritzLost]

@LostKobrakai Thanks for the clarifications! As for the PHP read access, I'm more worried about remote code execution through undiscovered vulnerabilities in external packages (or an attacker gaining write access to a package that gets included through the dependency chain, like we've seen with the event-stream debacle recently). If a PHP file with such a vulnerability lives inside the webroot (especially if it's in a "standard" path like /vendor/vulnerable-package/...), this could give an attacker access to the entire system. Granted, direct access can be blocked in the .htaccess or something like that, but for me it's just one less thing to worry about. Though I'm admittedly a bit paranoid ?

ProcessWire's classloader is certainly a good option as well, I just use the composer autoloader because I need to set up Composer anyway for most projects, I guess it comes down to preference. For a shared module or site profile I'd definitely go with the ProcessWire classloader!

[LostKobrakai]

13 hours ago, MoritzLost said:

Granted, direct access can be blocked in the .htaccess or something like that, but for me it's just one less thing to worry about. Though I'm admittedly a bit paranoid

That's what I was refering to. Afaik /vendor is not blocked by default just because there are packages out there containing assets, which need to be web-accessable. I'd personally also lean to just block it.

[bernhard]

Thx @MoritzLost!

Could you please explain the benefit of using composer compared to packing things into a pw module?

[horst]

@MoritzLost regarding earliest include of the autoloader, I think the site config.php is the right place. It is parsed / executed before modules, but it get not overwritten with core updates. ?

[LostKobrakai]

4 hours ago, bernhard said:

Could you please explain the benefit of using composer compared to packing things into a pw module?

For composer itself: interoperability and a far bigger ecosystem. For wrapping packages into pw modules: Please don't wrap packages for the sake of wrapping them. It's useful if you provide additional features like configuration through the modules or deeper integration in processwire classes, but that's work and needs to be maintained on top of the package by itself. The only backdraw a plain composer package has on top of a pw module without additional features, is that it can't be installed via the backend, but you need to use the composer cli (and that's more a restriction of composer than of pw). The same is actually also true for the few pw modules which install composer packages. 

Another problem of wrapping packages is that you cannot get updates without the wrapping module being updated. There is also lot's of tooling like checking for security issues around composer. For anyone not just doing php as a hobby I don't see a reason not to use composer just like nobody uses javascript without npm anymore for any serious work.

[gmclelland]

For those who wonder how they can have a module with the third party PHP library managed by Composer, see https://processwire.com/blog/posts/composer-google-calendars-and-processwire/

Also, here is the issue that talks about making Composer's vendor directory location configurable and possibly blocking the vendor/ folder via .htaccess: https://github.com/processwire/processwire-requests/issues/191

 

[MoritzLost]

@bernhard @LostKobrakai Just adding my two cents here ... I think if we're talking about writing site-specific modules or code, both options are equally valid and it comes down to preference. I try to write my site-specific functionality in general classes (not extending the Module class) and wire them up in my templates; in this case, it's just less work to connect the namespace through composer than to set up a module and add the namespace manually through the classloader. But really, for code that won't be used anywhere else it doesn't really matter. For reusable code or shareable modules I guess it depends on how closely it integrates with ProcessWire; i.e. my Textformatter module is built as a Processwire module, since it can't be used outside of Processwire anyway. But I'm also working on a collection of utility classes for my personal workflows (based on my markup generation tutorials), and most of those could be used in another context, so in this case I'm building it as a normal composer package, not as a ProcessWire module.

One usecase for the Composer autoloader for me is this module: It's not available on any public repository or the module directory, but I can still manually clone it into my project from my private repo and just connect the namespace through Composer. In this case it would definitely be more work to write a wrapper module around it, so for that it's a useful technique.

On 12/13/2018 at 1:57 PM, horst said:

@MoritzLost regarding earliest include of the autoloader, I think the site config.php is the right place. It is parsed / executed before modules, but it get not overwritten with core updates. ?

Thanks @horst, no idea why that never crossed my mind ? It's much better indeed, I've edited the tutorial accordingly!

On 12/13/2018 at 3:58 PM, gmclelland said:

For those who wonder how they can have a module with the third party PHP library managed by Composer, see https://processwire.com/blog/posts/composer-google-calendars-and-processwire/

I don't think that will work in this specific setup though, since the custom installer mentioned in that blogpost just returns "site/modules/..." as the install path, which isn't correct if your composer.json lives one directory above. Though it would be trivial to fork it and just add the public/ part there ...

[gmclelland]

Hi MoritzLost,

In case you didn't already know... You can also check out https://github.com/wanze/TemplateEngineTwig as a good example of a module that doesn't bundle a php library with the module.

In this case you can run: composer require wanze/template-engine-twig:^2.0 --no-dev

That one line would install the needed php libraries in the correct vendor folder and install the Processwire module in the correct directory.

[MoritzLost]

21 hours ago, gmclelland said:

Hi MoritzLost,

In case you didn't already know... You can also check out https://github.com/wanze/TemplateEngineTwig as a good example of a module that doesn't bundle a php library with the module.

In this case you can run: composer require wanze/template-engine-twig:^2.0 --no-dev

That one line would install the needed php libraries in the correct vendor folder and install the Processwire module in the correct directory.

Thanks! Though this does require running Composer in the project / web root, which I don't like, see the post above.

I had considered TemplateEngineTwig for a recent project when I wanted to work with Twig instead of pure PHP templates, but I ended up writing my custom integration, which makes use of the setup I described here. See my newest tutorial on that ^^

[VeiJari]

@MoritzLost Thanks for the tutorial!

I just have a major problem and I don't have a clue why this happens.

in site/config.php when i try to require the path to my autoload.php: require '../../libraries/vendor/autoload.php';

autoload is located at: websitex/libraries/vendor/autoload.php

Everything is white, every single page. I assume I'm getting some error on this instance, but I can't check it anywhere. When I comment it out, everything works (except the library I'm trying to use in the first place)

What should I do?

[teppo]

24 minutes ago, VeiJari said:

Everything is white, every single page. I assume I'm getting some error on this instance, but I can't check it anywhere. When I comment it out, everything works (except the library I'm trying to use in the first place)

What should I do?

Could be a simple issue with the paths. When site/config.php is loaded, the working directory should be site root, so you may be going way below the level you actually intended. You could try __DIR__ . '/../../libraries/vendor/autoload.php' instead, or just './libraries/vendor/autoload.php' (in case your libraries directory is in your site root – or ../ if it's below it).

[teppo]

On 12/14/2018 at 12:02 AM, MoritzLost said:

I don't think that will work in this specific setup though, since the custom installer mentioned in that blogpost just returns "site/modules/..." as the install path, which isn't correct if your composer.json lives one directory above. Though it would be trivial to fork it and just add the public/ part there ...

I know this is an old topic, but if you include wireframe-framework/processwire-composer-installer as a dependency for your project, it has a feature for overriding the default path in the root composer.json (which in my case is usually located in the site directory).

This isn't documented anywhere, but technically this should make it possible to install modules and site profiles to any directory of your choice (relative to current working directory):

{
  "require": {
    ...
  },
  "extra": {
    "pw-module-path": "site/modules",
    "pw-site-profile-path": "",
  }
}

Also what I found interesting about Composer installers is that even if a package, such as an existing module, requires something like hari/pw-module as a dependency, if you define another "competing" installer in the root composer.json it overrides the one defined in individual packages. Most modules use Hari's version of the installer, but that's not really an issue for me, since my root package defines the wireframe version as a dependency – and things just work.

Feels a bit hacky to me, but technically that's how it's supposed to go ?

[VeiJari]

20 hours ago, teppo said:

Could be a simple issue with the paths. When site/config.php is loaded, the working directory should be site root, so you may be going way below the level you actually intended. You could try __DIR__ . '/../../libraries/vendor/autoload.php' instead, or just './libraries/vendor/autoload.php' (in case your libraries directory is in your site root – or ../ if it's below it).

require './vendor/autoload.php'; this did the trick, when I desperately changed the folder name.

Thanks for you help!

[valan]

How to use both vendor libs and PW api in command line scripts? Assuming script is located here: prj/console/myscript.php

[MoritzLost]

1 hour ago, valan said:

How to use both vendor libs and PW api in command line scripts? Assuming script is located here: prj/console/myscript.php

@valan You need to include both the autoloader from Composer and the ProcessWire bootstrap file, see Bootstrapping ProcessWire CMS.  Assuming your autoloader lives under prj/vendor/autoload.php and the webroot with the ProcessWire installation under prj/web/, you can use the following at the top of your script:

# prj/console/myscript.php 

<?php
namespace ProcessWire;

# include composer autoloader
require __DIR__ . '/../vendor/autoload.php';
# bootstrap processwire
require __DIR__ . '/../web/index.php';

ProcessWire will detect that it is being included in this way and automatically load all the API variables (and the functions API, if you are using that). Keep in mind that there will be no $page variable, as there is no HTTP request, so there is no current page.

[Hector Nguyen]

@MoritzLost Do you encountered any issue with the autoloader? I used this inside public/site/init.php

require __DIR__ . '/../vendor/autoload.php';

But the autoloader cannot work properly, it is saying my namespace is undefined

2021-04-14 08:38:10	nntoan	http://xxx.test/	Fatal Error: 	Uncaught Error: Call to undefined function QFramework\Phrase() in /srv/users/capima/webapps/xxx/releases/1/public/site/templates/home.php:10 Stack trace: #0 /srv/users/capima/webapps/xxx/releases/1/public/wire/core/TemplateFile.php(318): require() #1 /srv/users/capima/webapps/xxx/releases/1/public/wire/core/Wire.php(394): ProcessWire\TemplateFile->___render() #2 /srv/users/capima/webapps/xxx/releases/1/public/wire/core/WireHooks.php(823): ProcessWire\Wire->_callMethod() #3 /srv/users/capima/webapps/xxx/releases/1/public/wire/core/Wire.php(465): ProcessWire\WireHooks->runHooks() #4 /srv/users/capima/webapps/xxx/releases/1/public/wire/modules/PageRender.module(536): ProcessWire\Wire->__call() #5 /srv/users/capima/webapps/xxx/releases/1/public/wire/core/Wire.php(397): ProcessWire\PageRender->___renderPage() #6 /srv/users/capima/webapps/xxx/releases/1/public/wire/core/WireHooks.php(823): ProcessWire\Wire->_callMethod() #7 /srv/users/capima/webapps/xxx/releases/1/public (line 10 of /srv/users/capima/webapps/xxx/releases/1/public/site/templates/home.php)

Any ideas?

[MoritzLost]

@Hector Nguyen Functions can't be autoloaded in PHP. Two options to work around this:

• Put the QFramework\Function function in a class as a static method, then the class can be autoloaded.
• Add all files containing functions in the autoload files list in your composer.json. This way those files will be included on every request.

[eydun]

Thank you for this write-up.

It is possible to also include processwire with composer? E.g.

composer require processwire/processwire

 

[wbmnfktr]

Have a look here:

https://processwire.com/docs/start/install/new/#installation-with-composer-and-packagist

[eydun]

@wbmnfktr thanks, I am aware of that.

My question meant to be, if it is possible to also include processwire with composer in the site-structure that @MoritzLost is recommending.

[MoritzLost]

@eydun Yeah, but it's a bit involved. ProcessWire expects its core files in the webroot and won't work properly if they're inside the vendor folder. What you can do is use Composer scripts to hook into the Composer installation and put all of ProcessWire's files in place. What I've done in the past is use a custom Composer script to download the latest version of ProcessWire from Github and unzip it in place. Something like this:

"prepare-installation": [
  "git clone --single-branch --branch dev https://github.com/processwire/processwire.git processwire-temp",
  "cp -r processwire-temp/wire public/wire",
  "cp -r processwire-temp/site-* public/",
  "cp processwire-temp/index.php public/",
  "cp processwire-temp/install.php public/",
  "cp processwire-temp/htaccess.txt public/.htaccess",
  "rm -rf processwire-temp",
],

This can be run with composer run-script prepare-installation. If you want this to happen automatically during composer install, use post-install-cmd instead of a custom script name. There's probably a simpler way to do this, this was for a project template with a couple of other caveats.

[eydun]

Thanks for the tip about composer scripts, that might come handy when setting up new processwire-installations.

Quote

ProcessWire expects its core files in the webroot and won't work properly if they're inside the vendor folder.

The docs mention:

composer require processwire/processwire

as a possible installation method: https://processwire.com/docs/start/install/new/#installation-with-composer-and-packagist

But I guess that is not feasible...

[MoritzLost]

On 3/9/2023 at 10:51 PM, eydun said:

But I guess that is not feasible...

@eydun Yeah, I don't think this has ever been tested or officially supported. It will definitely not work out of the box, as ProcessWire expects a specific folder structure and doesn't work if it's placed in the vendor folder. Not sure why this is even in the docs.