Leaderboard

Took a lot longer than anticipated to get this out of the oven, but version 0.29.0 of SearchEngine was just released. Word of warning: this is the biggest new release for the module so far (in terms of features as well as changed lines of code) so please test carefully! Main features included in this release: New group_by option in find_args. For now only supported values are null (default) and "template", the latter of which (obviously) groups results by template. Here grouping means two things: first of all results are rendered in tabs on the front-end, and second of all instead of a single Query object the module instead returns a QuerySet object, which in turn contains multiple Query objects. New results_grouped_by option in render_args. This accepts any field name, page property, etc. and essentially divides found results into groups by said field value / property. For an instance if you specify "template.label", you'll get a list of results with subheadings for templates. While both of these options are related to grouping results, they work on an entirely different level: group_by performs separate SQL queries and splits results into their own Query objects, while results_grouped_by is something that only affects the render phase (front-end). It's also possible to combine the two, in case that makes sense ? There's also a new custom sort setting "_indexed_templates" (results are sorted by the order of templates in the "indexed templates" config setting) and new argument "pinned_templates" for the find() method that will show results with provided templates at the top of the results list.

But I was ? And I tried to explain why... IMHO not That's what I was trying to explain. I'm using custom page classes in an object oriented way (similar to PW modules). And that's great because you have all your code pieces in one place - namely in one single php file of your custom page class. Let's say we build a webpage about cats. Step one: Create a custom page class and place it in /site/classes <?php namespace ProcessWire; class Cat extends Page { } Now all cats in our system are of type "Cat", and that's great! Now we want a custom headline for every cat on our frontend based on the title field, easy: <?php namespace ProcessWire; class Cat extends Page { public function headline() { if(!$this->title) return "This cat has no name"; return "Details for Cat '{$this->title}'"; } } In your template <h1><?= $page->headline() ?></h1> And that you'll do for any kind of business logic your application needs. Now what if we wanted to manage thousands of cats and we wanted every cat have a unique 4-letter page name instead of one based on the title (to prevent ugly suffix urls like catname-1, catname-2 etc)? We could do that easily with a hook. In the past one might have put the hook into ready.php and after time your ready.php grows and grows and grows and some day you totally lose control over your hooks and over your software. Not if you place them where they belong: Into the page class to the cat object: <?php namespace ProcessWire; class Cat extends Page { public function init() { $tpl = "template=cat"; $this->addHookAfter("Pages::saveReady($tpl,id=0)", $this, "onCreate"); } public function headline() { if(!$this->title) return "This cat has no name"; return "Details for Cat '{$this->title}'"; } public function onCreate($event) { $page = $event->arguments(0); $page->name = $event->pages->names()->uniqueRandomPageName(4); } } That's how you'd attach a hook in an autoload module, and that's what I suggest: Structure your custom page class just like an autoload pw module. Apply the same principles and your code will get better readable, better understandable and better maintainable. Now the only problem ist, that init() of the page class does not get called like it gets called in an autoload module. Neither does ready(). But it's simple to overcome this - just place this in init.php $cat = new Cat(); $cat->init(); This triggers the init method of your cat on INIT of PW and it attaches all hooks of your cat page class as if they where placed in init.php; The same technique can be used for ready() in ready.php; That does load one instance of cat into memory though. That's a little overhead. IMHO it's worth the overhead, but I can only guess that the overhead is very small... Maybe one could also use loaded() for that, I don't know and I'd have to investigate on this. Maybe someone can try and share their findings ?

A Fieldtype for dynamic options that are generated at runtime via a hook. Configuration Inputfield type You can choose from a range of inputfield types on the Details tab of a Dynamic Options field. Your field will return a string or an array depending on if the selected input field type is for "single item selection" or "multiple item selection". Maximum number of items You can define a maximum number of items the field is allowed to contain. The core inputfields supported by this module will become disabled once the limit is reached. This option is only applicable if you have selected an inputfield type that is for multiple item selection. Format as Pagefile/Pageimage object(s) If the field will store paths/URLs to Pagefiles/Pageimages then you can enable this option to have the formatted value be a Pagefile/Pageimage object for "single" fields or an array of Pagefile/Pageimage objects for "multiple" fields. There is a related Select Images inputfield module that allows you to visually select image thumbnails. Defining selectable options Selectable options for a Dynamic Options field should be set in a FieldtypeDynamicOptions::getSelectableOptions hook in /site/ready.php. The hook should return an array of options as 'value' => 'label'. An example hook is shown on the Details tab of a Dynamic Options field: $wire->addHookAfter('FieldtypeDynamicOptions::getSelectableOptions', function(HookEvent $event) { // The page being edited $page = $event->arguments(0); // The Dynamic Options field $field = $event->arguments(1); if($field->name === 'your_field_name') { $event->return = [ 'red' => 'Red', 'green' => 'Green', 'blue' => 'Blue', ]; } }); Formatted value If a Dynamic Options field uses a "single" input type then its formatted value is a string, and if it uses a "multiple" input type then its formatted value is an array. The unformatted value of a Dynamic Options field is always an array. Also see the Configuration section above for description of an option to have the formatted value be Pagefile/Pageimage object(s). Examples of possible uses $wire->addHookAfter('FieldtypeDynamicOptions::getSelectableOptions', function(HookEvent $event) { // The page being edited $page = $event->arguments(0); // The Dynamic Options field $field = $event->arguments(1); // Select from the "files" field on the page if($field->name === 'select_files') { $options = []; foreach($page->files as $file) { // Value is basename, label is description if one exists $options[$file->basename] = $file->get('description|basename'); } $event->return = $options; } // Select from files in a folder if($field->name === 'select_folder_files') { $options = []; $path = $event->wire()->config->paths->root . 'my-folder/'; $files = $event->wire()->files->find($path); foreach($files as $file) { // Value is full path, label is basename $options[$file] = str_replace($path, '', $file); } $event->return = $options; } // Select from non-system templates if($field->name === 'select_template') { $options = []; foreach($event->wire()->templates as $template) { if($template->flags & Template::flagSystem) continue; $options[$template->id] = $template->name; } $event->return = $options; } // Select from non-system fields if($field->name === 'select_field') { $options = []; foreach($event->wire()->fields as $field) { if($field->flags & Field::flagSystem) continue; $options[$field->id] = $field->name; } $event->return = $options; } // Select from FormBuilder forms if($field->name === 'select_formbuilder_form') { $form_names = $event->wire()->forms->getFormNames(); // Use form names as both keys and values $event->return = array_combine($form_names, $form_names); } }); https://github.com/Toutouwai/FieldtypeDynamicOptions https://processwire.com/modules/fieldtype-dynamic-options/

Needed a really simple solution to embed audio files within page content and couldn't find a module for that, so here we go. Textformatter Audio Embed works a bit like Textformatter Video Embed, converting this: <p>https://www.domain.tld/path/to/file.mp3</p> Into this: <audio controls class="TextformatterAudioEmbed"> <source src="https://www.domain.tld/path/to/file.mp3" type="audio/mpeg"> </audio> The audio element has pretty good browser support, so quite often this should be enough to get things rolling ? GitHub repository: https://github.com/teppokoivula/TextformatterAudioEmbed Modules directory: https://modules.processwire.com/modules/textformatter-audio-embed/

This week I've been working on something a little different for the core. Specifically, ProcessWire's database class (WireDatabasePDO) has been rewritten to support separate read-only and read-write database connections. Jan, who administers the processwire.com server, asked if I could implement it. I looked into it and thought it had significant benefits for ProcessWire users, so it worked out that now was a good time to implement it. The rewritten database class is actually complete and now running on a live test installation, but I have not yet committed it to the core dev branch because I want to fully document it first. This will happen next week in a blog post. By then I'll have processwire.com using it too. But I can already say that it's a cool thing watching the graphs in AWS show the difference it makes when we start hitting the site with crawlers. You might be wondering what the benefits are in having separate read-only and read-write database connections. I'll get into some of the details next week. But essentially, read-only database connections can scale in a way (and at much lower cost) than read-write connections can. And when using a service like Amazon Aurora, they can scale on-the-fly automatically according to traffic and demand for resources. Not only does it open up the ability for a ProcessWire-powered site to scale much further than before, but it has potential to reduce the costs of doing so by perhaps 50% or more. If you are interested in reading more, we are currently testing the features using Amazon Aurora and RDS Read Replicas (see also Replication with Aurora). However, the ProcessWire core support for this feature is not bound to anything AWS specific and will work with any platform supporting a similar ability. Thanks for reading, I'll have more on this next week, and also have it ready to use should you want to try it out yourself.

Not all the API variables and classes are included in the API Reference menus. For example, the whole $fieldgroups API variable and methods exist in the documentation but do not appear in the menus. Ryan has said: Personally I'd rather have everything listed in the menus. In terms of modules there is also the Pro API Explorer.

I'd go third party with this, I don't think there's anything similar to a forum "processwire native" aside from what @adrian mentions. I had once tested Vanilla Forums javascript integration and seemed super convenient as a plug and play solution. Although you get the overhead of hosting the forum software itself.

wow! processWire is just amazing, thank you

This sort of thing is possible, but you might find life easier with https://processwire.com/modules/search-engine/ - it will also be much more performant for the end user.

There is also this new module from @BitPoet

Thanks adrian, looks like a very good solution (as always if there is Tracy around ? ). I wonder if @ryan could evaluate to add all the dev branch additions and implementations right inside the website docs, tagging those to mark a distinction between them and those whom belong to the master branch.

Smart ?? Thanks!

You must have something else going on. I just tried it with a custom HomePage class in site/classes/HomePage.php <?php namespace ProcessWire; class HomePage extends Page { public function ready() { bd('ready!'); } } And in ready.php $p = new HomePage(); $p->ready(); And it properly dumps "ready!" to the tracy bar ?

I'd say that these are more like two sides of the same coin: You've solved the file issue by directing static file requests to CDN and pointing file related requests to a single (?) master instance. This is of course a valid approach, but in this type of setup you've got at least two types of instances (read and read-write), which can make things more complicated than just replicating a single instance type based on the load, and may not work so well for sites that often require direct access to assets. This could also potentially impact fault tolerance: in case you indeed only have one write instance, that going down for any reason would be a big blow. Most cloud setups I've worked with have handled this the other way around: files offloaded to external buckets and database shared among instances. One benefit of this is that those instances are identical, there's no "master" to rely on. In most cases S3 hasn't had major impact on performance (plus there's always CDN) and I've never run into a case where database would've been an absolute bottleneck. Of course DB requests should be limited: most web requests can be served from static cache, and if advanced caching is needed, in comes Redis. The long story short is that I believe core level ability to store files somewhere other than local disk would be a splendid improvement for cloud use. It's true that you can get past this and it's not strictly necessary in order to run ProcessWire in the cloud, but it would simplify things quite a bit. That being said, I definitely get what you mean by this being the more complex part ? Also, just to be clear, I think that the database abstraction is a great addition!

Hehe, There are days, and there are days like these. At 0.0.9 now, even refactored the code a bit and added syntax highlighting for embedded PHP snippets. I think I'll call it a day now though.

With help from Elabx, I'm now able to see it again on FF. The HTTP was being redirected to HTTPS so it was failing to load. Problem solved.

@MrSnoozles @teppo This is not a limiting factor in scalability at least. First off, at least here, the file-based assets are delivered by Cloudfront CDN, so they aren't part of the website traffic in the first place (other than to feed the CDN). If you wanted scalability then you'd likely want a CDN serving your assets whether using S3 or not. But a CDN isn't a necessary part of the equation in our setup either. File systems can be replicated just like databases. That's how this site runs on a load balancer on any number of nodes. Requests that upload files are routed to node A (primary), but all other requests can hit any node that the load balancer decides to route it to. The other nodes are exact copies of the node A file system that update in real time. This is very similar to what's happening with the DB reader (read-only) and writer (read-write) connection I posted about above, where the writer would be node A and there can be any number of readers. Something like S3 doesn't enhance the scalability of any of this. Implementing S3 as a storage option for PW is still part of the plan here, but more for the convenience and usefulness than scalability. You can already use S3 as a storage option in PW if you use one of the methods of mapping directories in your file system to it. But I'm looking to support for it in PW more natively than that. It is admittedly more complex than the DB stuff mentioned above. For instance, we use PHP's PDO class that all DB requests go through, so intercepting and routing them is relatively simple. Whereas in PHP, there is no PDO-type class that the file system API is built around, and instead it is dozens of different procedural functions (which is just fine, until you need to change their behavior). In addition, calls to S3 are more expensive than a file system access, so doing something as simple as getting an image dimensions is no longer a matter of a simple php getimagesize() call. Instead, it's more like making an FTP connection somewhere, FTP'ing the file to your computer, getting the image dimensions, storing them somewhere else, then deleting the image. So meta data like image dimensions needs to be stored somewhere else. PW actually implemented this ability last year (meta data and stored image dimensions). So we've already been making small steps towards S3-type storage, but because the big picture is still pretty broad in scope to implement, it's more of a long term plan. Though maybe one of my clients will tell me they need it next week, in which case it'll become a short term plan. ?

I don't understand what you mean by "frontend" page classes ?

In Page.php we have the hookable loaded method: /** * For hooks to listen to, triggered when page is loaded and ready * * #pw-hooker * */ public function ___loaded() { } It is called on these spots in PW: So this means that loaded() is triggerd on a page find operation: I'm not 100% sure if that means that it is called on every page find or if PW's internal cache somehow plays a role, but as long as the page is not loaded, the loaded hook will not trigger. If you look at the docs of modules you see that init() is called immediately when the module is loaded and ready() is called when the api is ready. This can make an important difference sometimes where hooks work in init() but not in ready() and vice versa. Page objects don't have these methods though. These are all the hookable methods of the Page baseclass: Additionally you have hookable methods in PagePermissions.module: And several others ? But no init() and ready(). Because init() and ready() are a concept of PW modules: My point is that loaded() of a page might only be triggered in very special cases. Imagine a module that loads a page only on uninstall() of the module. Then the loaded() event of your pageClass would happen very late in the request. But if you use custom page classes in an object oriented way then they become much more like a PW module and then I think it makes sense to stick to the PW module naming convention and that means that init() is triggered on module/pageclass init - just like you had the code in init.php. That IMHO does also mean, that this code does get executed on EVERY request. It does NOT mean, that the code is executed only ONCE - that's a matter of the "singular" setting of a module. See examples below. Code in ready() on the other hand is just like code in ready.php - it fires once, but later in the process, when the API is ready and all other init() methods of all modules and pageclasses have been triggered. Try this /site/modules/Test.module.php <?php namespace ProcessWire; class Test extends WireData implements Module { public static function getModuleInfo() { return [ 'title' => 'Test', 'version' => '0.0.1', 'summary' => 'Your module description', 'autoload' => false, 'singular' => false, 'icon' => 'smile-o', 'requires' => [], 'installs' => [], ]; } public function init() { bd('test init'); } public function ready() { bd('test ready'); } } Install the module and then call it from the tracy console: It only fires init() but not ready() - because when loading the module PW's ready event has already been triggered. Now make that module autoload, do a modules refresh and fire the console code again: You see the difference? ? Now just for fun we make the module "singular": If I happen to need something to execute on every load of the pageClass I use the class constructor: This means that my custom pageclass always has the correct template and does always live under the correct parent in the pagetree. As you can see I define all my fields as class constants. This makes the code so much more readable and also makes typos nearly impossible. It might feel like overhead, but actually it makes me a lot more efficient as well: So I'm actually using most if not all of my page classes as something like a singular autoload module. Overhead? Maybe a little. Problem? I think/hope no ? Hope that helps. Suggestions for improving this further always welcome ?

Thanks for the module PWaddict, very useful for my current use case. I'm building a site with numerous roles, none of which have access to the admin panel as everything is done through the API and front-end editing. I had to enable view access for some templates, so the only 'look-in' to the top level page tree is through a trusted user adding a link in CKEditor fields. It's nice to have a module that focuses simply on the main page tree, so I can still define page reference fields for hidden templates where the role still has view access, and they display fine.

RF3 got a very nice little update today ? Aggregations Often we need to calculate sums or averages of table data quickly and efficiently. RockFinder3 makes that easy as well: $avg = $rockfinder->find("template=cat") ->addColumn('weight') ->getObject("SELECT AVG(weight)"); db($avg); $cats = $rockfinder ->find("template=cat") ->addColumns(['title', 'weight']); $cats->dump(); // dump finder data to tracy $obj = $cats->getObject("SELECT SUM(`weight`) AS `total`, COUNT(`id`) AS `cats`, SUM(`weight`)/COUNT(`id`) AS `avg`"); db($obj); // dump result of aggregation What happens behind the scenes is that RockFinder3 gets the current SQL query of the finder and adds that as FROM (...sql...) AS tmp to the query that you provide for aggregation. This is the resulting SQL query of the example above: SELECT SUM(`weight`) AS `total`, COUNT(`id`) AS `cats`, SUM(`weight`)/COUNT(`id`) AS `avg` FROM ( SELECT `pages`.`id` AS `id`, `_field_title_605cab16f38ce`.`data` AS `title`, `_field_weight_605cab16f3993`.`data` AS `weight` FROM `pages` LEFT JOIN `field_title` AS `_field_title_605cab16f38ce` ON `_field_title_605cab16f38ce`.`pages_id` = `pages`.`id` LEFT JOIN `field_weight` AS `_field_weight_605cab16f3993` ON `_field_weight_605cab16f3993`.`pages_id` = `pages`.`id` WHERE (pages.templates_id=44) AND (pages.status<1024) GROUP BY pages.id ) AS tmp You can even provide a suffix for your query to do things like GROUP BY etc: $rf = $rockfinder->find("template=cat|dog"); $rf->addColumns(['created']); $rf->dump(); db($rf->getObjects( "SELECT COUNT(id) AS `count`, DATE_FORMAT(created, '%Y-%m-%d') AS `date`", "GROUP BY DATE_FORMAT(created, '%Y-%m-%d')" ));