Leaderboard

I've been working with ProcessWire for a while now, and I've noticed that using Composer to manage dependencies and autoload external libraries isn't as prevalent in ProcessWire development as in other areas of PHP programming. I started out by using the default setup recommend in this blogpost. However, one major problem I have with this approach is that all external dependencies live in the webroot (the directory the server points to), which is unfavourable from a security standpoint and, in my opinion, just feels a bit messy. In this tutorial, I want to go through a quick setup of Composer and ProcessWire that keeps the dependencies, all custom-written code and other source material outside of the webroot, and makes full usage of the Composer autoloader. This setup is pretty basic, so this tutorial is probably more useful to beginners (this is why I'll also include some general information on Composer), but hopefully everyone can take something away from this for their personal workflow. Site structure after setup This is what the directory structure can look like after the setup: . ├── composer.json ├── composer.lock ├── node_modules │ └── ... ├── public │ ├── index.php │ ├── site │ ├── wire │ └── ... ├── packacke-lock.json ├── package.json ├── sass │ ├── main.scss │ ├── _variables.scss │ └── ... ├── src │ ├── ContentBag.php │ └── ... └── vendor ├── autoload.php ├── composer ├── league ├── symfony └── ... As mentioned, the main point of this setup is to keep all external libraries, all other custom source code and resources out of the webroot. That includes Composer's vendor folder, your node_modules and JavaScript source folder if you are compiling JavaScript with webpack or something similar and including external scripts via NPM, or your CSS preprocessor files if you are using SASS or LESS. In this setup, the public directory acts as the webroot (the directory that is used as the entry point by the server, DocumentRoot in the Apache configuration). So all other files and directories in the mysite folder aren't accessible over the web, even if something goes wrong. One caveat of this setup is that it's not possible to install ProcessWire modules through Composer using the PW Module Installer (see Blogpost above), but that's just a minor inconvenience in my experience. Installation You'll need to have composer installed on your system for this. Installation guides can be found on getcomposer.org. First, open up your shell and navigate to the mysite folder. $ cd /path/to/mysite/ Now, we'll initialize a new Composer project: $ composer init The CLI will ask some questions about your projects. Some hints if you are unsure how to answer the prompts: Package names are in the format <vendor>/<project>, where vendor is your developer handle. I use my Github account, so I'll put moritzlost/mysite (all lowercase). Project type is project if you are creating a website. Author should be in the format Name <email>. Minimum Stability: I prefer stable, this way you only get stable versions of dependencies. License will be proprietary unless you plan on sharing your code under a FOSS license. Answer no to the interactive dependencies prompts. This creates the composer.json file, which will be used to keep track of your dependencies. For now, you only need to run the composer install command to initialize the vendor directory and the autoloader: $ composer install Now it's time to download and install ProcessWire into the public directory: $ git clone https://github.com/processwire/processwire public If you don't use git, you can also download ProcessWire manually. I like to clean up the directory after that: $ cd public $ rm -r .git .gitattributes .gitignore CONTRIBUTING.md LICENSE.TXT README.md Now, setup your development server to point to the /path/to/mysite/public/ directory (mind the public/ at the end!) and install ProcessWire normally. Including & using the autoloader With ProcessWire installed, we need to include the composer autoloader. If you check ProcessWire's index.php file, you'll see that it tries to include the autoloader if present. However, this assumes the vendor folder is inside the webroot, so it won't work in our case. One good place to include the autoloader is using a site hook file. We need the autoloader as early as possible, so we'll use init.php: EDIT: As @horst pointed out, it's much better to put this code inside the config.php file instead, as the autoloader will be included much earlier: // public/site/config.php <?php namespace Processwire; require '../../vendor/autoload.php'; The following also doesn't apply when including the autoloader in the config-file. This has one caveat: Since this file is executed by ProcessWire after all modules had their init methods called, the autoloader will not be available in those. I haven't come across a case where I needed it this early so far; however, if you really need to include the autoloader earlier than that, you could just edit the lines in the index.php file linked above to include the correct autoloader path. In this case, make sure not to overwrite this when you update the core! Now we can finally include external libraries and use them in our code without hassle! I'll give you an example. For one project, I needed to parse URLs and check some properties of the path, host et c. I could use parse_url, however that has a couple of downsides (specifically, it doesn't throw exceptions, but just fails silently). Since I didn't want to write a huge error-prone regex myself, I looked for a package that would help me out. I decided to use this URI parser, since it's included in the PHP League directory, which generally stands for high quality. First, install the dependency (from the project root, the folder your composer.json file lives in): $ composer require league/uri-parser This will download the package into your vendor directory and refresh the autoloader. Now you can just use the package in your own code, and composer will autoload the required class files: // public/site/templates/basic-page.php <?php namespace Processwire; use \League\Uri\Parser; // ... if ($url = $page->get('url')) { $parser = new Parser(); $parsed_url = $parser->parse($url); // do stuff with $parsed_url ... } Wiring up custom classes and code Another topic that I find really useful but often gets overlooked in Composer tutorials is the ability to wire up your own namespace to a folder. So if you want to write some object-oriented code outside of your template files, this gives you an easy way to autoload those using Composer as well. If you look at the tree above, you'll see there's a src/ directory inside the project root, and a ContentBag.php file inside. I want to connect classes in this directory with a custom namespace to be able to have them autoloaded when I use them in my templates. To do this, you need to edit your composer.json file: { "name": "moritzlost/mysite", "type": "project", "license": "proprietary", "authors": [ { "name": "Moritz L'Hoest", "email": "info@herebedragons.world" } ], "minimum-stability": "stable", "require": {}, "autoload": { "psr-4": { "MoritzLost\\MySite\\": "src/" } } } Most of this stuff was added during initialization, for now take note of the autoload information. The syntax is a bit tricky, since you have to escape the namespace seperator (backslash) with another backslash (see the documentation for more information). Also note the PSR-4 key, since that's the standard I use to namespace my classes. The line "MoritzLost\\MySite\\": "src/" tells Composer to look for classes under the namespace \MoritzLost\MySite\ in the src/ directory in my project root. After adding the autoload information, you have to tell composer to refresh the autoloader information: $ composer dump-autoload Now I'm ready to use my classes in my templates. So, if I have this file: // src/ContentBag.php <?php namespace MoritzLost\MySite; class ContentBag { // class stuff } I can now use the ContentBag class freely in my templates without having to include those files manually: // public/site/templates/home.php <?php namespace Processwire; use MoritzLost\MySite\ContentBag; $contentbag = new ContentBag(); // do stuff with contentbag ... Awesome! By the way, in PSR-4, sub-namespaces correspond to folders, so I can put the class MoritzLost\MySite\Stuff\SomeStuff in src/Stuff/SomeStuff.php and it will get autoloaded as well. If you have a lot of classes, you can group them this way. Conclusion With this setup, you are following secure practices and have much flexibility over what you want to include in your project. For example, you can just as well initialize a JavaScript project by typing npm init in the project root. You can also start tracking the source code of your project inside your src/ directory independently of the ProcessWire installation. All in all, you have good seperation of concerns between ProcessWire, external dependencies, your templates and your OOP-code, as well as another level of security should your Server or CGI-handler ever go AWOL. You can also build upon this approach. For example, it's good practice to keep credentials for your database outside the webroot. So you could modify the public/site/config.php file to include a config or .env file in your project root and read the database credentials from there. Anyway, that's the setup I came up with. I'm sure it's not perfect yet; also this tutorial is probably missing some information or isn't detailed enough in some areas depending on your level of experience. Feel free to ask for clarification, and to point out the things I got wrong. I like to learn as well ? Thanks for making it all the way to the bottom. Cheers!

I think this may be the problem. 58 seems to be the ID of your template, but the config setting must be supplied with the ID of the page. In your example from the opening post, that page is “Members”, so you want to change Members’ template to owners (58) and put Members’ ID into $config->usersPageIDs. It’s probably a four digit number rather than a double digit template ID.

You're very welcome, @szabesz! (and thank you!) This thread started with my how-to questions about image migration. Besides the modules that @adrian and @kongondo mentioned, I didn't find any clear documentation about how to deal with image migration with comprehensive detail (I might have missed finding it). Of course, I read a number of pages about image scripting, but I didn't see info that covered all the bases. Adrian expressed skepticism about the stability of the module Kongondo mentioned, and Adrian's module didn't fit my specific needs, so I didn't use either of those modules, and decided to proceed with my own script. But my script didn't deal with cropped images (because I didn't have any), unless they're automatically brought over by the copy command I ran to bring in any image variations. Thus, my script might have missed some important things with images. I'm not sure. I'd love to see a how-to on this very topic, i.e. a comprehensive write-up of how to migrate images and catch every detail. My efforts were a "roll-your-own" attempt to do it, and it worked for me. But I have no confidence that I covered all the details of image migration. If I did, I'd love to hear about it. ? Peter

Go here and star the repo) P.S. And, of course, keep the love to ProcessWire always burning deep in you heart. But do star the repo too, to make your inner feelings make a a change in the world!

@Mamun Paul, Welcome to the forums and ProcessWire. Please don't double post. You asked the same question here and @adrian provided a response. Thanks.

I have an external php script (launched by cron) that uses ProcessWire (i.e. by including index.php, as explained here). I have the ProcessRedirectIds module installed, and when I run the script I get this warning: Notice: Undefined index: REQUEST_URI in /var/www/www.mysite.example/site/assets/cache/FileCompiler/site/modules/ProcessRedirectIds/ProcessRedirectIds.module on line 66 Obviously there is no REQUEST_URI in this case, but the module seems not aware of this possibility...

This error indicates that you have not added your site's domain to the allowed referers for your Google Maps API key. See the error documentation here: https://developers.google.com/maps/documentation/javascript/error-messages#deverrorcodes This didn't use to be required, but Google Maps is constantly becoming more strict with their API usage limits. Go to the cloud console and make sure to add your site's domain to the allowed referers for the key you're using, then it should start working again.

Thank you @huseyin, I got it working with the suggested Ajax Search module and some minor tweaking with inspiration from your site and ProcessWire.com. In the end I decided not to use the live search anyway. But I got to try it out, and it could be useful for future clients.

Hi All, I've added routines to the script to create a file of hrefs in the body field, and also a file of 301 redirects for an .htaccess file. There are so many edge cases with internal hrefs that I decided to edit them by hand, based on the list created. (For example, an href that links to a second internal PW page on the target domain that PW's url system on the source domain doesn't know about.) I also moved the chown command to include pages that didn't have images, because the file dir has to be owned by the domain and running the script as root ends up making the file dir owned by root. Again, I realize that this script may only serve a very small group of people who need to do exactly what I needed it for: to migrate many sections of pages from one PW domain to another PW domain that had a different section structure. I've successfully and quickly moved many sections and pages as of now, so I'm confident that the script is doing what I needed. If someone else finds it useful, that's grand. ? Here it is, below. Click the "reveal hidden contents" icon below to see it. Peter

Was thinking more of these, because I've seen all the github issues.

Just an FYI for anyone who reads this - this is deprecated since MySQL 8: https://dev.mysql.com/doc/refman/8.0/en/information-functions.html#function_found-rows I have told Ryan here: https://github.com/processwire/processwire-issues/issues/1146#issuecomment-629428382

Well its just a simple listing of content, I basically I was just wanted to give RockFinder a shot to see if it performed faster and it does! Basically brings down the time to nothing. Took a dive into the code and notice how you strip down the SQL select, I am guessing that's why it peforms faster since the original db query includes SQL_COUNT_FOUND_ROW. So, I'm starting to think I have no way to solve this without counting? Unless I make a prev/next pagination only maybe? Of course, I can always cache all of this, but just wanted to see if I could squeeze out a bit more of performance from the raw request.

The view in the screenshot is what you should get by calling <?= $modules->get('SearchEngine')->render() ?>. If you mean that you get the search form and a list of results but the results don't have a summary field, then it's possible that you need to define the summary field — this is assuming that your pages don't actually have a filled-in summary field on them: $config->SearchEngine = [ 'render_args' => [ // Summary of each result (in the search results list) is the value of this field. 'result_summary_field' => 'summary', ], ]; These are all translatable strings — so no, manual rendering is not needed, you just need to translate the module files to your language ? The array part is interesting. perhaps options field (if I understood the next part correctly and this is what you're using here) isn't getting indexed quite right; I'll take a closer look at that soon. The code I posted a few replies ago on this thread was actually almost directly applicable to your case as well. Here's a rough idea — you still need to fill in the blanks, most importantly the way you're fetching those products: $wire->addHookAfter('SearchEngine::savedPageIndex', function(HookEvent $event) { $page = $event->arguments[0]; if ($page->template == 'TemplateWithProducts') { $searchEngine = $event->modules->get('SearchEngine'); // I don't know exactly how you're fetching those products; include your logic here :) $products = $pages->find('template=product, some_rule=' . $page->some_field); foreach ($products as $product) { $product_index = $searchEngine->indexPage($product, false, [ 'return' => 'index', ]); $page->search_index .= "\n" . $product_index[0]; } $page->save('search_index', [ 'quiet' => true, 'noHooks' => true, ]); } }); Note that if this is a multi-lingual site, that will require some additional tweaks (indexPage() will return an index value per language, and you need to use setLanguageValue() to store it). Let me know if any of this doesn't seem quite right, and I'll be happy to take a closer look ?

I would probably prefer to put my users under the Members page directly. That way you would still end up with your suggested page tree, but those pages would actually be the users and there would be no need to maintain a relationship between users and their pages. Have a look at this blog post for a tutorial: https://processwire.com/blog/posts/processwire-core-updates-2.5.14/#multiple-templates-or-parents-for-users

@ryan Thank you for your insight, it was an interesting read to learn more about your current refactoring efforts. I am wondering if it is the right time for you to also start working on the Javascript API perhaps? I don't remember since when it is on the roadmap, but in this post from 2017 you wrote: "...so I'm going to continue to keep this in our roadmap."

I've seen a couple of questions regarding namespaces and autoloading floating around the forum recently, so I decided to write a little tutorial. In general, I often see people getting confused when they try to wrap their head around namespaces, autoloading, Composer and the mapping of namespaces to directory structures all at once. In fact, those are very much independent, distinct concepts, and it is much easier to explain and understand them separately. So this guide is structured as follows: How namespaces work in PHP. How autoloading works in PHP. Conventions for mapping namespaces to directory structures: PSR-4. How autoloading works in Composer and ProcessWire's class loader. How to use the class loader in a ProcessWire module. Feel free to skip the sections you're already familiar with. Namespaces in PHP The purpose of namespaces in PHP is to avoid naming conflicts between classes, functions and constants, especially when you're using external libraries and frameworks. Nothing more. It's important to understand that this has nothing at all to do with autoloading, directory structures or file names. You can put namespaced stuff everywhere you want. You can even have multiple namespaces inside a single file (don't try this at home). Namespaces only exist to be able to use a generic name – for example, ProcessWire's Config class – multiple times in different contexts without getting a naming conflict. Without namespaces, I couldn't use any library that includes a Config class of it's own, because that name is already taken. With namespaces, you can have a distinction between the classes ProcessWire\Config and MoritzLost\Config. You can also use sub-namespaces to further segregate your code into logical groups. For example, I can have two classes MoritzLost\Frontend\Config and MoritzLost\Backend\Config– a class name only needs to be unique within it's namespace. You can declare the namespace for a PHP file using the namespace statement at the top: // file-one.php <?php namespace ProcessWire; // file-two.php <?php namespace MoritzLost\Frontend; This way, all classes, methods and constants defined inside this file are placed in that namespace. All ProcessWire classes live in the ProcessWire namespace. Now to use one of those classes – for example, to instantiate it – you have a couple of options. You can either use it's fully qualified class name or import it into the current namespace. Also, if you are inside a namespaced file, any reference to a class is relative to that namespace. Unless it starts with a backward slash, in this case it's relative to the global namespace. So all of those examples are equivalent: // example-one.php <?php namespace ProcessWire; $page = new Page(); // example-two.php <?php use ProcessWire\Page; $page = new Page(); // example-three.php <?php $page = new ProcessWire\Page(); // example-four.php <?php namespace MoritzLost\Somewhere\Over\The\Rainbow; $page = new \ProcessWire\Page(); The use statement in the second example can be read like this: “Inside this file, all references to Page refer to the class \ProcessWire\Page” How autoloading works Every PHP program starts with one entry file – for ProcessWire, that's usually it's index.php. But you don't want to keep all your code in one file, that would get out of hand quickly. Once you start to split your code into several individual files however, you have to take care of manually including them with require or include calls. That becomes very tedious as well. The purpose of autoloading is to be able to add new code in new files without having to import them manually. This, again, has nothing to do with namespaces, not even something with file locations. Autoloading is a pretty simple concept: If you try to use a class that hasn't been loaded yet, PHP calls upon it's registered autoloaders as a last-ditch attempt to load them before throwing an exception. Let's look at a simple example: // classes.php <?php class A { /** class stuff */ } class B { /** class stuff */ } // index.php <?php spl_autoload_register(function ($class) { include_once 'classes.php'; }); new A(); new B(); This is a complete and functional autoloader. If you don't believe me, go ahead and save those two files (classes.php and index.php) and run the index.php with php -f index.php. Then comment out the include_once call and run it again, then you'll get an error that class A was not found. Now here's what happens when index.php is executed (with the autoloader active): Our anonymous function is added to the autoload queue through spl_autoload_register. PHP tries to instantiate class A, but can't because it's not loaded yet. If there was no autoloader registered, the program would die with a fatal error at this point. But since there is an autoloader ... The autoloader is called. Our autoloader includes classes.php with the class definition. That was a close one! Since the class has been loaded, execution goes back to the index.php which can now proceed to instantiate A and B. If the class was still not loaded at this point, PHP would go back to the original plan and die. One thing to note is that the autoloader will only be called once in this example. That's because both A and B are in the same file and that file is included during the first call to the autoloader. Autoloading works on files, not on classes! The important takeaway is that PHP doesn't know if the autoloader knows where to find the class it asks for or, if there are multiple autoloader, which one can load it. PHP just calls each registered autoloader in turn and checks if the class has been loaded after each one. If the class still isn't loaded after the last autoloader is done, it's error time. What the autoloader actually does is pretty much wild wild west as well. It takes the name of the class PHP is trying to load as an argument, but it doesn't have to do anything with it. Our autoloader ignores it entirely. Instead, it just includes classes.php and says to itself “My job here is done”. If class A was in another file, it wouldn't have worked. This process has two main advantages: Since autoloaders are only called on-demand to load classes just in time, we only include the files we actually need. If in the example above class A and B are not used in some scenarios, the classes.php will not be included, which will result in better performance for larger projects (though this isn't as cut and dry, since autoloading has it's own overhead, so if you load most classes anyway during a single request, it will actually be less efficient). If the autoloader is smart enough to somehow map class names to the files they're located in, we can just let the autoloader handle including the classes we need, without having to worry about jamming include statements everywhere. That brings us to ... PSR-4, namespaces and directory structures As you see, namespaces and autoloading are both pretty limited concepts. And they aren't inherently linked to each other. You can namespace your classes without ever adding an autoloader, and you can autoload classes that are all in the same namespace. But they become useful when you put them together. At the core of all that autoloading talk is a simple idea: By putting classes in files named after their class names, and putting those files in directory hierarchies based on the namespace hierarchy, the autoloader can efficiently find and load those files based on the namespace. All it needs is a list of root namespaces with their corresponding directories. The exact way class names and namespaces are mapped to directory structures and file names is purely conventional. The accepted convention for this is PSR-4. This is a super simple standard which basically just sums up the ideas above: A base namespace is mapped to a specific directory in the file system. When the autoloader is asked to load a class in that namespace (or a sub-namespace of it), it starts looking in that folder. This "base" namespace may include multiple parts – for example, I could use MoritzLost\MyAwesomeLibrary as a base and map that to my source directory. PSR-4 calls this a "namespace prefix". Each sub-namespace corresponds to a sub-directory. So by looking at the namespace, you can follow subdirectories to the location where you expect to find the class file. Finally, the class name is mapped directly to the file name. So MyCoolClass needs to be put inside MyCoolClass.php. This all sounds simple and straightforward - and it absolutely is! It's only once you mash everything together, mix up language features, accepted conventions and proprietary implementations like Composer on top that it becomes hard to grasp in one go. Composer and ProcessWire's class loader Now all that's left is to talk about how Composer and ProcessWire provide autoloading. Composer, of course, is primarily a tool for dependency management. But because most libraries use namespaces and most developers want to have the libraries they're using autoloaded, those topics become a prerequisite to understanding what Composer does in this regard. Composer can use different autoloading mechanisms; for example, you can just give it a static list of files to include for every request, or use the older PSR-0 standard. But most modern libraries use PSR-4 to autoload classes. So all Composer needs to function is a mapping of namespace prefixes to directories. Each library maintains this mapping for it's PSR-4-structured classes through the autoload information in their composer.json. You can do this for your own site to: Just include the autoload information as shown in the documentation and point it to the directory of your class files. Composer collects all that information and uses it to generate a custom file at vendor/autoload.php — that's the one you need to include somewhere whenever you set up Composer in one of your projects. Bells and whistles aside, this file just registers an autoloader function that will use all the information collected from your own and your included libraries' composer.json to locate and include class files on demand. You can read more about how to optimize Composer's autoloader for production usage here. If you want to read up on how to set up Composer for your own sites, read my ProcessWire + Composer integration guide instead. And finally, what does ProcessWire do to handle all this? Turns out, ProcessWire has it's own autoloader implementation that is more or less PSR-4 compliant. You can access it as an API variable ($classLoader or wire('classLoader'), depending on context). Instead of using a static configuration file like Composer, the namespace -> directory mapping is added during the runtime by calling $classLoader->addNamespace. As you would expect, this function accepts a namespace and a directory path. You can use this to register your own custom namespaces. Alternatively, if you have site-specific classes within the ProcessWire namespace, you can just add their location to the class loader using the same method: $classLoader->addNamespace('ProcessWire', '/path/to/your/classes/'). Utilizing custom namespaces and autoloading in ProcessWire modules Now as a final remark, I wanted to give an example of how to use custom namespaces and the class loader in your own modules. I'll use my TrelloWire module as an example: Decide what namespace you're going to use. The main module file should live in the ProcessWire namespace, but if you have other classes in your module, they can and should use a custom namespace to avoid collisions with other modules. TrelloWire uses ProcessWire\TrelloWire, but you can also use something outside the ProcessWire namespace. You need to make sure to add the namespace to the class loader as early as possible. If either you or a user of your module tries to instantiate one of your custom classes before that, it will fail. Good places to start are the constructor of your main module file, or their init or ready methods. Here's a complete example. The module uses only one custom namespaced class: ProcessWire\TrelloWire\TrelloWireApi, located in the src/ directory of the module. But with this setup, I can add more classes whenever I need without having to modify anything else. /** * The constructor registers the TrelloWire namespace used by this module. */ public function __construct() { $namespace = 'ProcessWire\\TrelloWire'; $classLoader = $this->wire('classLoader'); if (!$classLoader->hasNamespace($namespace)) { $srcPath = $this->wire('config')->paths->get($this) . 'src/'; $classLoader->addNamespace($namespace, $srcPath); } } Source Thanks for making it through to the very end! I gotta learn to keep those things short. Anyway, I hope this clears up some questions about namespaces and autoloading. Let me know if I got something wrong, and feel free to add your own tips and tricks!

have you tried to double click one trash icon? Should do the trick. (since PW version 2.2?)

If you have the Version Control module installed it might be the issue linked to below, which has been fixed in an update to the module:

For more advanced uses (api or even json-api by specification) I'd recommend something like fractal.