Jump to content
MoritzLost

How to setup Composer and use external libraries in ProcessWire

Recommended Posts

I've been working with ProcessWire for a while now, and I've noticed that using Composer to manage dependencies and autoload external libraries isn't as prevalent in ProcessWire development as in other areas of PHP programming. I started out by using the default setup recommend in this blogpost. However, one major problem I have with this approach is that all external dependencies live in the webroot (the directory the server points to), which is unfavourable from a security standpoint and, in my opinion, just feels a bit messy.

In this tutorial, I want to go through a quick setup of Composer and ProcessWire that keeps the dependencies, all custom-written code and other source material outside of the webroot, and makes full usage of the Composer autoloader. This setup is pretty basic, so this tutorial is probably more useful to beginners (this is why I'll also include some general information on Composer), but hopefully everyone can take something away from this for their personal workflow.

Site structure after setup

This is what the directory structure can look like after the setup:

.
├── composer.json
├── composer.lock
├── node_modules
│   └── ...
├── public
│   ├── index.php
│   ├── site
│   ├── wire
│   └── ...
├── packacke-lock.json
├── package.json
├── sass
│   ├── main.scss
│   ├── _variables.scss
│   └── ...
├── src
│   ├── ContentBag.php
│   └── ...
└── vendor
    ├── autoload.php
    ├── composer
    ├── league
    ├── symfony
    └── ...

As mentioned, the main point of this setup is to keep all external libraries, all other custom source code and resources out of the webroot. That includes Composer's vendor folder, your node_modules and JavaScript source folder if you are compiling JavaScript with webpack or something similar and including external scripts via NPM, or your CSS preprocessor files if you are using SASS or LESS. In this setup, the public directory acts as the webroot (the directory that is used as the entry point by the server, DocumentRoot in the Apache configuration). So all other files and directories in the mysite folder aren't accessible over the web, even if something goes wrong.

One caveat of this setup is that it's not possible to install ProcessWire modules through Composer using the PW Module Installer (see Blogpost above), but that's just a minor inconvenience in my experience.

Installation

You'll need to have composer installed on your system for this. Installation guides can be found on getcomposer.org.

First, open up your shell and navigate to the mysite folder.

$ cd /path/to/mysite/

Now, we'll initialize a new Composer project:

$ composer init

The CLI will ask some questions about your projects. Some hints if you are unsure how to answer the prompts:

  • Package names are in the format <vendor>/<project>, where vendor is your developer handle. I use my Github account, so I'll put moritzlost/mysite (all lowercase).
  • Project type is project if you are creating a website.
  • Author should be in the format Name <email>.
  • Minimum Stability: I prefer stable, this way you only get stable versions of dependencies.
  • License will be proprietary unless you plan on sharing your code under a FOSS license.
  • Answer no to the interactive dependencies prompts.

This creates the composer.json file, which will be used to keep track of your dependencies. For now, you only need to run the composer install command to initialize the vendor directory and the autoloader:

$ composer install

Now it's time to download and install ProcessWire into the public directory:

$ git clone https://github.com/processwire/processwire public

If you don't use git, you can also download ProcessWire manually. I like to clean up the directory after that:

$ cd public $ rm -r .git .gitattributes .gitignore CONTRIBUTING.md LICENSE.TXT README.md

Now, setup your development server to point to the /path/to/mysite/public/ directory (mind the public/ at the end!) and install ProcessWire normally.

Including & using the autoloader

With ProcessWire installed, we need to include the composer autoloader. If you check ProcessWire's index.php file, you'll see that it tries to include the autoloader if present. However, this assumes the vendor folder is inside the webroot, so it won't work in our case.

One good place to include the autoloader is using a site hook file. We need the autoloader as early as possible, so we'll use init.php:

EDIT: As @horst pointed out, it's much better to put this code inside the config.php file instead, as the autoloader will be included much earlier:

// public/site/config.php
<?php
namespace Processwire;

require '../../vendor/autoload.php';

The following also doesn't apply when including the autoloader in the config-file.

This has one caveat: Since this file is executed by ProcessWire after all modules had their init methods called, the autoloader will not be available in those. I haven't come across a case where I needed it this early so far; however, if you really need to include the autoloader earlier than that, you could just edit the lines in the index.php file linked above to include the correct autoloader path. In this case, make sure not to overwrite this when you update the core!

Now we can finally include external libraries and use them in our code without hassle! I'll give you an example. For one project, I needed to parse URLs and check some properties of the path, host et c. I could use parse_url, however that has a couple of downsides (specifically, it doesn't throw exceptions, but just fails silently). Since I didn't want to write a huge error-prone regex myself, I looked for a package that would help me out. I decided to use this URI parser, since it's included in the PHP League directory, which generally stands for high quality.

First, install the dependency (from the project root, the folder your composer.json file lives in):

$ composer require league/uri-parser

This will download the package into your vendor directory and refresh the autoloader.

Now you can just use the package in your own code, and composer will autoload the required class files:

// public/site/templates/basic-page.php
<?php
namespace Processwire;

use \League\Uri\Parser;

// ...
if ($url = $page->get('url')) {
	$parser = new Parser();
	$parsed_url = $parser->parse($url);
	// do stuff with $parsed_url ...
}

Wiring up custom classes and code

Another topic that I find really useful but often gets overlooked in Composer tutorials is the ability to wire up your own namespace to a folder. So if you want to write some object-oriented code outside of your template files, this gives you an easy way to autoload those using Composer as well. If you look at the tree above, you'll see there's a src/ directory inside the project root, and a ContentBag.php file inside. I want to connect classes in this directory with a custom namespace to be able to have them autoloaded when I use them in my templates.

To do this, you need to edit your composer.json file:

{
    "name": "moritzlost/mysite",
    "type": "project",
    "license": "proprietary",
    "authors": [
        {
            "name": "Moritz L'Hoest",
            "email": "info@herebedragons.world"
        }
    ],
    "minimum-stability": "stable",
    "require": {},
	"autoload": {
		"psr-4": {
			"MoritzLost\\MySite\\": "src/"
		}
	}
}

Most of this stuff was added during initialization, for now take note of the autoload information. The syntax is a bit tricky, since you have to escape the namespace seperator (backslash) with another backslash (see the documentation for more information). Also note the PSR-4 key, since that's the standard I use to namespace my classes.

The line "MoritzLost\\MySite\\": "src/" tells Composer to look for classes under the namespace \MoritzLost\MySite\ in the src/ directory in my project root. After adding the autoload information, you have to tell composer to refresh the autoloader information:

$ composer dump-autoload

Now I'm ready to use my classes in my templates. So, if I have this file:

// src/ContentBag.php
<?php
namespace MoritzLost\MySite;

class ContentBag {
	// class stuff
}

I can now use the ContentBag class freely in my templates without having to include those files manually:

// public/site/templates/home.php
<?php
namespace Processwire;

use MoritzLost\MySite\ContentBag;

$contentbag = new ContentBag();
// do stuff with contentbag ...

Awesome!

By the way, in PSR-4, sub-namespaces correspond to folders, so I can put the class MoritzLost\MySite\Stuff\SomeStuff in src/Stuff/SomeStuff.php and it will get autoloaded as well. If you have a lot of classes, you can group them this way.

Conclusion

With this setup, you are following secure practices and have much flexibility over what you want to include in your project. For example, you can just as well initialize a JavaScript project by typing npm init in the project root. You can also start tracking the source code of your project inside your src/ directory independently of the ProcessWire installation. All in all, you have good seperation of concerns between ProcessWire, external dependencies, your templates and your OOP-code, as well as another level of security should your Server or CGI-handler ever go AWOL. You can also build upon this approach. For example, it's good practice to keep credentials for your database outside the webroot. So you could modify the public/site/config.php file to include a config or .env file in your project root and read the database credentials from there.

Anyway, that's the setup I came up with. I'm sure it's not perfect yet; also this tutorial is probably missing some information or isn't detailed enough in some areas depending on your level of experience. Feel free to ask for clarification, and to point out the things I got wrong. I like to learn as well 🙂

Thanks for making it all the way to the bottom. Cheers!

Edited by MoritzLost
Include autoloader in config.php instead of init.php
  • Like 10
  • Thanks 8

Share this post


Link to post
Share on other sites

Just to clarify the bit about "putting stuff outside the webroot" for security reasons: This is certainly a nice precaution, but processwire is not using it because there are way to many shared hosters, where it's simply not possible to go outside the webroot. I'd also say that if your webserver is not behaving correctly you've got bigger problems to deal with, so usually just adjusting your .htaccess file should be perfectly fine as well. Especially as your php process still needs to be able to read things even if they're not within the webroot to be able to execute them.

3 hours ago, MoritzLost said:

Another topic that I find really useful but often gets overlooked in Composer tutorials is the ability to wire up your own namespace to a folder.

There's not really a need for composer for that part as processwire has it's own psr-4 compatible classloader: https://processwire.com/api/ref/class-loader/

  • Like 5
  • Thanks 1

Share this post


Link to post
Share on other sites

@LostKobrakai Thanks for the clarifications! As for the PHP read access, I'm more worried about remote code execution through undiscovered vulnerabilities in external packages (or an attacker gaining write access to a package that gets included through the dependency chain, like we've seen with the event-stream debacle recently). If a PHP file with such a vulnerability lives inside the webroot (especially if it's in a "standard" path like /vendor/vulnerable-package/...), this could give an attacker access to the entire system. Granted, direct access can be blocked in the .htaccess or something like that, but for me it's just one less thing to worry about. Though I'm admittedly a bit paranoid 😁

ProcessWire's classloader is certainly a good option as well, I just use the composer autoloader because I need to set up Composer anyway for most projects, I guess it comes down to preference. For a shared module or site profile I'd definitely go with the ProcessWire classloader!

  • Like 5

Share this post


Link to post
Share on other sites
13 hours ago, MoritzLost said:

Granted, direct access can be blocked in the .htaccess or something like that, but for me it's just one less thing to worry about. Though I'm admittedly a bit paranoid

That's what I was refering to. Afaik /vendor is not blocked by default just because there are packages out there containing assets, which need to be web-accessable. I'd personally also lean to just block it.

  • Like 1

Share this post


Link to post
Share on other sites

@MoritzLost regarding earliest include of the autoloader, I think the site config.php is the right place. It is parsed / executed before modules, but it get not overwritten with core updates. 😀

  • Like 3
  • Thanks 1

Share this post


Link to post
Share on other sites
4 hours ago, bernhard said:

Could you please explain the benefit of using composer compared to packing things into a pw module?

For composer itself: interoperability and a far bigger ecosystem. For wrapping packages into pw modules: Please don't wrap packages for the sake of wrapping them. It's useful if you provide additional features like configuration through the modules or deeper integration in processwire classes, but that's work and needs to be maintained on top of the package by itself. The only backdraw a plain composer package has on top of a pw module without additional features, is that it can't be installed via the backend, but you need to use the composer cli (and that's more a restriction of composer than of pw). The same is actually also true for the few pw modules which install composer packages. 

Another problem of wrapping packages is that you cannot get updates without the wrapping module being updated. There is also lot's of tooling like checking for security issues around composer. For anyone not just doing php as a hobby I don't see a reason not to use composer just like nobody uses javascript without npm anymore for any serious work.

  • Like 4

Share this post


Link to post
Share on other sites

For those who wonder how they can have a module with the third party PHP library managed by Composer, see https://processwire.com/blog/posts/composer-google-calendars-and-processwire/

Also, here is the issue that talks about making Composer's vendor directory location configurable and possibly blocking the vendor/ folder via .htaccess: https://github.com/processwire/processwire-requests/issues/191

 

  • Like 1

Share this post


Link to post
Share on other sites

@bernhard @LostKobrakai Just adding my two cents here ... I think if we're talking about writing site-specific modules or code, both options are equally valid and it comes down to preference. I try to write my site-specific functionality in general classes (not extending the Module class) and wire them up in my templates; in this case, it's just less work to connect the namespace through composer than to set up a module and add the namespace manually through the classloader. But really, for code that won't be used anywhere else it doesn't really matter. For reusable code or shareable modules I guess it depends on how closely it integrates with ProcessWire; i.e. my Textformatter module is built as a Processwire module, since it can't be used outside of Processwire anyway. But I'm also working on a collection of utility classes for my personal workflows (based on my markup generation tutorials), and most of those could be used in another context, so in this case I'm building it as a normal composer package, not as a ProcessWire module.

One usecase for the Composer autoloader for me is this module: It's not available on any public repository or the module directory, but I can still manually clone it into my project from my private repo and just connect the namespace through Composer. In this case it would definitely be more work to write a wrapper module around it, so for that it's a useful technique.

On 12/13/2018 at 1:57 PM, horst said:

@MoritzLost regarding earliest include of the autoloader, I think the site config.php is the right place. It is parsed / executed before modules, but it get not overwritten with core updates. 😀

Thanks @horst, no idea why that never crossed my mind 😅 It's much better indeed, I've edited the tutorial accordingly!

On 12/13/2018 at 3:58 PM, gmclelland said:

For those who wonder how they can have a module with the third party PHP library managed by Composer, see https://processwire.com/blog/posts/composer-google-calendars-and-processwire/

I don't think that will work in this specific setup though, since the custom installer mentioned in that blogpost just returns "site/modules/..." as the install path, which isn't correct if your composer.json lives one directory above. Though it would be trivial to fork it and just add the public/ part there ...

  • Like 3

Share this post


Link to post
Share on other sites

Hi MoritzLost,

In case you didn't already know... You can also check out https://github.com/wanze/TemplateEngineTwig as a good example of a module that doesn't bundle a php library with the module.

In this case you can run: composer require wanze/template-engine-twig:^2.0 --no-dev

That one line would install the needed php libraries in the correct vendor folder and install the Processwire module in the correct directory.

  • Thanks 1

Share this post


Link to post
Share on other sites
21 hours ago, gmclelland said:

Hi MoritzLost,

In case you didn't already know... You can also check out https://github.com/wanze/TemplateEngineTwig as a good example of a module that doesn't bundle a php library with the module.

In this case you can run: composer require wanze/template-engine-twig:^2.0 --no-dev

That one line would install the needed php libraries in the correct vendor folder and install the Processwire module in the correct directory.

Thanks! Though this does require running Composer in the project / web root, which I don't like, see the post above.

I had considered TemplateEngineTwig for a recent project when I wanted to work with Twig instead of pure PHP templates, but I ended up writing my custom integration, which makes use of the setup I described here. See my newest tutorial on that ^^

Share this post


Link to post
Share on other sites

@MoritzLost Thanks for the tutorial!

I just have a major problem and I don't have a clue why this happens.

in site/config.php when i try to require the path to my autoload.php: require '../../libraries/vendor/autoload.php';

autoload is located at: websitex/libraries/vendor/autoload.php

Everything is white, every single page. I assume I'm getting some error on this instance, but I can't check it anywhere. When I comment it out, everything works (except the library I'm trying to use in the first place)

What should I do?

Share this post


Link to post
Share on other sites
24 minutes ago, VeiJari said:

Everything is white, every single page. I assume I'm getting some error on this instance, but I can't check it anywhere. When I comment it out, everything works (except the library I'm trying to use in the first place)

What should I do?

Could be a simple issue with the paths. When site/config.php is loaded, the working directory should be site root, so you may be going way below the level you actually intended. You could try __DIR__ . '/../../libraries/vendor/autoload.php' instead, or just './libraries/vendor/autoload.php' (in case your libraries directory is in your site root – or ../ if it's below it).

  • Like 2

Share this post


Link to post
Share on other sites
On 12/14/2018 at 12:02 AM, MoritzLost said:

I don't think that will work in this specific setup though, since the custom installer mentioned in that blogpost just returns "site/modules/..." as the install path, which isn't correct if your composer.json lives one directory above. Though it would be trivial to fork it and just add the public/ part there ...

I know this is an old topic, but if you include wireframe-framework/processwire-composer-installer as a dependency for your project, it has a feature for overriding the default path in the root composer.json (which in my case is usually located in the site directory).

This isn't documented anywhere, but technically this should make it possible to install modules and site profiles to any directory of your choice (relative to current working directory):

{
  "require": {
    ...
  },
  "extra": {
    "pw-module-path": "site/modules",
    "pw-site-profile-path": "",
  }
}

Also what I found interesting about Composer installers is that even if a package, such as an existing module, requires something like hari/pw-module as a dependency, if you define another "competing" installer in the root composer.json it overrides the one defined in individual packages. Most modules use Hari's version of the installer, but that's not really an issue for me, since my root package defines the wireframe version as a dependency – and things just work.

Feels a bit hacky to me, but technically that's how it's supposed to go 🙂

Share this post


Link to post
Share on other sites
20 hours ago, teppo said:

Could be a simple issue with the paths. When site/config.php is loaded, the working directory should be site root, so you may be going way below the level you actually intended. You could try __DIR__ . '/../../libraries/vendor/autoload.php' instead, or just './libraries/vendor/autoload.php' (in case your libraries directory is in your site root – or ../ if it's below it).

require './vendor/autoload.php'; this did the trick, when I desperately changed the folder name.

Thanks for you help!

  • Like 1

Share this post


Link to post
Share on other sites

How to use both vendor libs and PW api in command line scripts? Assuming script is located here: prj/console/myscript.php

Share this post


Link to post
Share on other sites
1 hour ago, valan said:

How to use both vendor libs and PW api in command line scripts? Assuming script is located here: prj/console/myscript.php

@valan You need to include both the autoloader from Composer and the ProcessWire bootstrap file, see Bootstrapping ProcessWire CMS.  Assuming your autoloader lives under prj/vendor/autoload.php and the webroot with the ProcessWire installation under prj/web/, you can use the following at the top of your script:

# prj/console/myscript.php 

<?php
namespace ProcessWire;

# include composer autoloader
require __DIR__ . '/../vendor/autoload.php';
# bootstrap processwire
require __DIR__ . '/../web/index.php';

ProcessWire will detect that it is being included in this way and automatically load all the API variables (and the functions API, if you are using that). Keep in mind that there will be no $page variable, as there is no HTTP request, so there is no current page.

  • Like 1

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By VeiJari
      Hello forum, this is my first security related post, so I'm a bit of a newbie.
      I understand that when I have direct front-input from user I should sanitize the input, but how about when I use a secret key for showing a API for a third-party supplier? Should I sanitize the input->get() key?
      I've tested this issue and I tried ?key=<?php echo $page->field; ?> And without adding any sanitization it comes back: /?key=<?php%20echo%20$page->field;%20?>
      So can I rely on this, or should I still use $sanitizer just in case?
       
      Thanks for the help!
    • By anttila
      We have many booking calendars made with ProcessWire (own databases) and I want to do a web app (SQL) which allows user to log in. First, the user chooses the right calendar and then (s)he have to log in. The user can be from any of those calendars and the app is not running on ProcessWire (it can if necessary). So if there any way to make sure that the user has rights to the calendar (s)he tries to log in and if the password is correct.
      Is there any better way to do this? I could also use PIN codes or something, but those need to be encrypted too.
      Multiple ProcessWires A lot of users per ProcessWire Everyone can log in to the web app (when using right calendar)
    • By Jennifer Stock
      Greetings. I would like to restrict access to certain sections of my organization's ProcessWire site using pubcookie. We are rolling out Shibboleth authentication later this year but for now, it seems I can only make use of our institution's single sign-on routine by utilizing rules in an .htaccess file. 
      I am wondering if there is a way to ask PW to apply these rules to certain pages in the site, whether via template type or location in the page tree:
      AuthType UWNetID PubcookieAppID "MyApplication" require type staff faculty  
    • By flydev
      Presentation
      Originaly developped by Jeff Starr, Blackhole is a security plugin which trap bad bots, crawlers and spiders in a virtual black hole.
      Once the bots (or any virtual user!) visit the black hole page, they are blocked and denied access for your entire site.
      This helps to keep nonsense spammers, scrapers, scanners, and other malicious hacking tools away from your site, so you can save precious server resources and bandwith for your good visitors.
       
      How It Works
      You add a rule to your robots.txt that instructs bots to stay away. Good bots will obey the rule, but bad bots will ignore it and follow the link... right into the black hole trap. Once trapped, bad bots are blocked and denied access to your entire site.

      The main benefits of Blackhole include:
       Bots have one chance to obey your site’s robots.txt rules. Failure to comply results in immediate banishment.
       
      Features
      Disable Blackhole for logged in users Optionally redirect all logged-in users Send alert email message Customize email message Choose a custom warning message for bad bots Show a WHOIS Lookup informations Choose a custom blocked message for bad bots Choose a custom HTTP Status Code for blocked bots Choose which bots are whitelisted or not  
      Instructions
      Install the module Create a new page and assign to this page the template "blackhole" Create a new template file "blackhole.php" and call the module $modules->get('Blackhole')->blackhole(); Add the rule to your robot.txt Call the module from your home.php template $modules->get('Blackhole')->blackhole();  Bye bye bad bots!

      Downloads
      https://github.com/flydev-fr/Blackhole http://modules.processwire.com/modules/blackhole/  
      Screen

       

       Enjoy
    • By godmok
      Hi everyone,
      I can't get my head around this, so maybe someone can help: I want to know how install the dev branch of processwire with composer.
      I use laragon and the "quick create" function and there you can run composer like this: "composer create-project processwire/processwire %s". OK, this is working, so I will get the master branch from it, but how about dev? I found some examples for laravel on this question, but no luck with it. Here some examples:
      composer create-project processwire/processwire %s --prefer-dist --stability=dev composer create-project processwire/processwire:dev %s composer create-project processwire/processwire:dev %s --stability=dev composer create-project processwire/processwire:dev %s --prefer-dist --stability=dev composer create-project processwire/processwire:dev-master %s --prefer-dist --stability=dev composer create-project processwire/processwire:dev-branch %s --prefer-dist --stability=dev composer create-project processwire/processwire:@dev %s --prefer-dist --stability=dev and so on... Maybe someone got this already and would like to share?
×
×
  • Create New...