Leaderboard

This week we have some improvements to the $pages->findRaw() method. Most of these are based on requested improvements in either GitHub (like #427) or the forums: Support for selecting parent properties/fields in the return value, i.e. $pages->findRaw("selector", "parent.title"); // Example 1 $pages->findRaw("selector", [ 'title', 'parent.title', 'parent.parent.title' ]); // Example 2 Support for selecting page 'meta' data in the return value, i.e. $pages->findRaw("selector", "meta"); // example 1 $pages->findRaw("selector", [ "meta.foo", "meta.bar" ]); // example 2 Support for selecting "references" in the return value. References are other pages that reference the found page(s) via Page reference fields, just like the $page->references() method. $pages->findRaw("selector", "references"); // example 1 $pages->findRaw("selector", [ 'references.title', 'references.url' ]); // example 2 If you want the references to be indexed by field name, just include "references.field" in the requested fields: $pages->findRaw("selector", [ 'references.field', 'references.title', 'references.url' ]); Support for selecting title and/or value from options (FieldtypeOptions) fields, i.e. $pages->findRaw("selector", [ 'my_options_field.*' ]); // example 1 $pages->findRaw("selector", [ 'my_options_field.title', 'my_options_field.value' ]); // example 2 Support for a new "flat" option that flattens multidimensional arrays in return value. This is best explained by an example. Let's say we did this: $pages->findRaw("categories.count>0, limit=1", "title, categories.title" ); And it returns an array similar to this for every matching page: [ 'title' => '2-factor authentication coming to ProcessWire', 'categories' => [ 1035 => [ 'title' => 'Security' ], 1288 => [ 'title' => 'Users' ] ] ] Now let's try the flat option: $pages->findRaw("categories.count>0, limit=1", "title, categories.title", [ 'flat' => true ]); (side note: you might already know this, but fields and options can also be bundled in the selector like this below, which is identical to the above): $pages->findRaw("categories.count>0, fields=title|categories.title, flat=1"); It flattens any multi-dimensional arrays in the return value to be indexed in the same way that you requested them in the $fields argument: [ 'title' => '2-factor authentication coming to ProcessWire', 'categories.title' => [ 1035 => 'Security', 1288 => 'Users' ] ] In addition to these updates for the $pages->findRaw() method, this week there have also been updates to the $sanitizer->float() method and InputfieldFloat module. Now both support numbers with E notation such as 1E-2, 10E-14, -1.253E-5, etc. There are also some new values for the getString option on the float sanitizer. I don't work with this kind of stuff very often, so thanks to @MetaTunes on GitHub for helping to figure it out. Lastly, the ProcessTemplate module also went through a refactoring with some minor improvements this week. Thanks for reading, have a great weekend!

Very good points there and definitely a big thumbs up for this topic in general, but not sure if I can fully agree with this part: Any module that supports Hari's composer installer, Wireframe composer installer, or the official composer installers package will automatically be installed in the correct directory. Technically only requirements are that the module includes composer.json, and is marked as pw-module or processwire-module. If you're using a custom directory structure (like we do), you'll indeed need a config setting in your main composer.json to specify where your modules directory is located — but even then it's a once-per-project thing. Hari's installer doesn't support this but others do, and Wireframe installer will also automatically detect any module's using Hari's installer and override it to make this work. On a loosely related note, in case module authors are reading this: Hari's installer may still be the most popular option, but it's somewhat outdated. It lacks support for Composer 2, doesn't provide any configuration settings, etc. I would strongly recommend moving to more recent (and actively maintained) alternatives. I always add the modules directory to Git ignore, so it's definitely doable. This works out of the box for composer installed modules, and modules that are not installed via composer can still be added to the repository with the -f flag, which essentially makes Git ignore the ignore rule. Only problem would be modules added in a production environment via GUI, but since we're talking about version controlling stuff, this doesn't seem like a particularly relevant concern to me. In case of modules that don't support Composer installation, creating a private fork is of course always an option. This works nicely for commercial modules as well, but requires installing said modules via Git repository (which is a bit clunky), or via private Packagist (which is more elegant, but requires paid plan.) In my setup ProcessWire is installed and updated via Composer. Again doable, but this part could admittedly be more elegant: since the wire directory needs to be in a publicly accessible location, typically you'll need post-install-cmd/post-update-cmd in your composer.json (or something similar) to move / copy said directory from vendor to another location. Anyway, would definitely like to see this topic getting more attention in future releases. And it would of course be great if more module authors would add their modules to Packagist — that would make things easier for Composer users, and setting this up is quite straightforward. ? In WP plugin installs via Composer are — for the most part — solved by wpackagist.org, which mirrors all plugins in the official plugins directory. In a nutshell this means that plugin authors just need to submit their plugins to the directory, and they automatically become installable via Composer (composer install wpackagist-plugin/plugin-name). Having something like that for PW would be an interesting option for sure.

Padloper 2 is Here

This week we have ProcessWire 3.0.192 on the dev branch. This is about 20 commits ahead of the previous version and contains 11 issue fixes and 5 pull requests, along with various other updates. PR code contributors in this version including Daun, MrSnoozles, Radon8472, BernhardBaumrock and Tyde. Thanks! For more details see the dev branch commit log. In addition to core updates, in recent weeks I've also been working on a new module similar to the PageAutosave (developed for the admin) except that it is for front-end forms. It works with any front-end form (whether home brewed or from something like FormBuilder) and remembers the entered form values every time a field is changed. If the user doesn't submit the form, they get an email with a link to finish filling out the form after a configurable period of time. This is especially nice for order and reservation forms. Accompanying the module is a separate Process module that lets you view all the forms in progress in the admin. Each form entry has its own unique URL, making them easy for users to return to, from any device. Once the user submits the form, the data gets removed from the pending submissions database. This is something I've been meaning to develop for awhile, and with the admin PageAutosave module still fresh on my mind, I thought now was a good time to do it. Thanks for reading, Happy New Year and have a great weekend!

Lister Native Date Format Allows the date format for "created", "modified" and "published" columns to be set within a Lister or Lister Pro instance. Why? Lister formats the "created", "modified" and "published" columns as relative time strings and doesn't provide an easy way to change this without writing code. Sometimes a value like "3 weeks ago" is not precise enough and you want to see an exact date/time in these columns, perhaps only temporarily before switching back to a relative date. Details An icon is added near the top right of the Lister that reveals a dropdown where you can select from a list date formats that you define in the module config. The Lister will remember your chosen date format the next time you visit the Lister. If you have Lister Pro installed then each Lister Pro instance will remember the date format previously set for that instance. Configuration In the "Date options" field in the module config, enter a list of date formats that are compatible with wireDate(), one format per line. The default date format used by Lister is "rel". You can also set a default date format for native fields which will apply to all Listers that haven't yet had a date format chosen from the dropdown. https://github.com/Toutouwai/ListerNativeDateFormat https://processwire.com/modules/lister-native-date-format/

Side note @ryan it would be nice if you could add relevant keywords to the title of new news or blog posts. This makes it easier to find information when you are searching for it weeks or months later ("weekly update 14.1." is not nearly as helpful as "weekly update find raw parents, sanitizer float" in a search result) +1 for better version control support. I've not built RockMigrations just for fun ?

@ryan, thanks for this week's updates! @MoritzLost has sparked an interesting discussion about using PW with version control and he makes a comparison with Craft CMS. A couple of excerpts from his comments: Ryan, it would be great to get your voice in the discussion. We're at the beginning of the new year so what do you think about putting better compatibility with version control onto a roadmap for 2022?

@elabx Craft CMS ? I'm also trying to move towards static site generator and headless and/or git-based CMS (like Forestry or Netlify CMS). My personal websites are built with SSG and hosted for free on Netlify (processwire.dev & mehrlicht.photos) – though those don't use a CMS at all. But most of the tools for the SSG/Jamstack approach are built to accomodate version control from the ground up. I've been eyeing Prismic, hopefully I'll get a project soon where I can try that out. Though it's hard to find clients for static sites + CMS right now, most have never heard of it … @benbyf Drupal 8/9 has that as well, Configuration Synchronization, but it's a pain to use (like Drupal itself). Luckily I migrated the last Drupal project I had to actively support away from Drupal and to Craft CMS last year ? @bernhard Thanks a lot for the detailed explanation and all the code examples! For the next bigger ProcessWire project, if there is one, I'll be sure to use RockMigrations. Sounds like it solves my problem regarding project schema / config well. But there's still a cost associated with this – every developer needs to know how to use it and not shoot themselves in the foot. How does a new developer install and work on the project from the repository? Is it as simple as composer install, php craft install, npm ci, npm run build? Because every additional step is something we don't have to do when using a CMS with first-party git support. For example, how do you install ProcessWire in this case? Manually? That's work I don't have to do if I can just composer install. Sure, I can hook something up to install ProcessWire using composer scripts on install (in fact, I have done something like this for our starter template which is under version control). But now I have a custom script that I need to maintain, fix if it breaks. Or I put the core in my version control, which means I have to litter my version history with core files. (BTW I could use git submodules but somehow nobody knows they're a thing ?) There are more issues like this as I mentioned (e.g. the issue regarding default selected pages in page reference fields; folders containing a mix of files that should and shouldn't be in version control; modules that don't use composer, …). Every one of those can be solved. But every solve either comes with some trade-off (like keeping not project related files in git) or with a maintenance cost and learning curve for a custom solution. And that's a hard sell if there are systems that don't require all that work. Beyond that, I have zero guarantees from the core that all those custom solutions will still work with the next core version (the lack of semantic versioning for the ProcessWire core doesn't help either). There are also side-effects to there not being explicit support from the core regarding those setups – for example, if we want to outsource something to an external developer or train a new hire, we can't just point them to the ProcessWire docs, but have to teach 'the way' to them. For Craft projects, we just follow the standards. Besides the problems I mentioned, there are also advantages that come from the fact that Craft supports version control explicitly and is geared towards it. For example, the production site will not allow any configuration changes (new fields, setting changes, etc) by default. So I have a guarantee that that the configuration will be exactly as I left it so I can deploy with confidence from the repository. With ProcessWire, I don't have a guarantee that nobody has changed some field setting in production while I was on holiday. Of course, not changing stuff in production can be established as a rule, but it's again something to be consciously aware of and enforce, instead of having the system enforce it. I feel like there isn't one single answer to this debate. I hear you – it's easier to complain than work on solutions. Not every project needs a totally 'clean' version control. Trade-offs are acceptable and arguably what we are paid for. I think there's room for multiple CMS with maybe slightly different focus. Right now I'm in the planning stage for a new ProcessWire project that I'm looking forward to. It's just a matter of using different tools depending on the job, no need for a single CMS to rule them all ? Not sure I understand the problem; with Craft, the project config is always the source of truth and the interface is just window dressing to edit the project config. If you're using external modules, they have to be built to support project config, but most plugins are by now. I have said that myself ? After having supported a larger project with regular updates for a while now, which is a bit of a pain. Maybe it would go better if we'd used RockMigrations from the start! Hindsight is 20/20 … @Robin S I'll try to get around to that comparison between Craft CMS and ProcessWire, which would include all the issues regarding version control. But knowing me and my tendency of feature-creep for articles that might take a while ?

I added a config field for this in v0.1.1. Let me know if this wasn't what you were getting at.

Did a quick test:

This would be some kind of a dream for me to have in ProcessWire. ?‍♂️

Please do, I'd be very interested to read that! It seems like this should be doable via a module in PW. I have a work-in-progress module that does something like this. It's not complete and haven't looked at it a while but it seemed like a promising avenue at the time. I think if the concept could be illustrated in a module then Ryan might see the benefits and then hopefully incorporate the features into the core. Seeing as you've clearly spent some time thinking about these issues, would you mind writing up a wishlist of what things you think PW should handle via files for the sake of version control? Fields and templates are the obvious things but you also mentioned translations which I wouldn't have thought of (I only work on single language websites) so it would be great to have a comprehensive list of problem areas for version control. Thanks in advance! ?

Sorry I missed your second post... Actually something similar would already be possible using RockMigrations. At least for basic setups. I'm not sure how that can work for more complex setups though. I'll try to explain that by an example: Let's say we created a new blog module. We need a blog page (parent) and several blog-items (children): - home '- blog |- blog entry 1 |- blog entry 2 ... This setup could easily be built into a module using rockmigrations: <?php namespace ProcessWire; class MyBlog extends WireData implements Module { public static function getModuleInfo() { return [ 'title' => 'MyBlog', 'version' => '0.0.1', 'summary' => 'blog module using rockmgirations', 'autoload' => true, 'singular' => true, 'icon' => 'smile-o', 'requires' => [], 'installs' => [], ]; } public function init() { $this->rm()->fireOnRefresh($this, "migrate"); } public function migrate() { $this->rm()->migrate([ 'fiels' => [...], 'templates' => [ 'blog_parent' => [...], 'blog_item' => [...], ], ]); $this->rm()->setParentChild('blog_parent', 'blog_item'); // --> this sets the family settings for both the parent and the child template - how long does that take by hand? ;) It means that all pages having the "blog_parent" template will create a "blog_item" page automatically when you click on "add new page". Basically the stuff that you set on the family tab of the template. } /** * @return RockMigrations */ public function rm() { return $this->wire->modules->get('RockMigrations'); } } That means I'm literally defining my config in the module code. We could also have the config read from a config.yaml - that would really be the same (just less powerful IMHO)! The problem arises when you want to make that config customizable. For example you want a page reference field to select the author for every blog post. Building that into the blog module would probably not be a good idea because not every site might need such a field. What I do in such situations is this: I make the migrate() method hookable and create one module that handles all the project-specific stuff: <?php namespace ProcessWire; class MyProject extends WireData implements Module { public static function getModuleInfo() { return [ 'title' => 'MyProject', 'version' => '0.0.1', 'summary' => 'module that handles site specific stuff', 'autoload' => true, 'singular' => true, 'icon' => 'smile-o', 'requires' => [], 'installs' => [], ]; } public function init() { $this->addHookAfter("MyBlog::migrate", $this, "migrate"); } public function migrate(HookEvent $event) { $this->rm()->createField("author", "page", [ 'label' => 'Select the author', ... ]); $this->rm()->addFieldToTemplate("author", "blog_item", "title"); } /** * @return RockMigrations */ public function rm() { return $this->wire->modules->get('RockMigrations'); } } That means you can have your blog module that you can reuse from project to project (and improve it and just pull changes if you need to) and you can still customize it the ProcessWire way (which we all love I guess). BUT Now we got a new field on the "blog_item", the "author" field. So the config.yaml of the blog module would change, wouldn't it? I'm not sure how CraftCMS handles such situations? And that's one of the reasons why I think that efforts like this are going in a wrong direction. It sounds intriguing for sure to have something like a "migrations recorder". I'm working or at least thinking about that since 2016. But I simply can't imagine a way that could reliably work in all situations. RockMigrations does. And it does it really well. And it does it for almost 3 years now and has evolved a lot. It's still not perfect for sure. But I don't think that This feels wrong. I'dont know how to explain. I think you have a good understanding of composer and such things. Likely a much better understanding than I have. So if it is true what you are saying, I must be doing a lot wrong in my daily work or at least I must be understanding a lot wrong... Because a lot of my workflows do feel very version control friendly. And I've lately started working in a company in vienna where we work on ProcessWire projects with fully automated workflows. We work in a team. We work on different local machines. We deploy to staging and if everything works (which has been the case 100% so far), we deploy to production with a single click. But maybe I'll change my opinion some day and look back and laugh. Maybe I should just try CraftCMS. Maybe many other fancy new tools. At my company they also work (or worked) with another CMS. They said they liked to be able to define fields and templates in code. They said that was one of the (few) things they didn't like about ProcessWire, because they love professional workflows and automated deployments... But they also admitted, that it was somewhat painful to define fields and templates in the other CMS. They said you have to add pieces of config in that file, then change another piece in another file. And end up changing many files for simple changes. Now they like RockMigrations ? And such comments make me think I am on a right track and we as ProcessWire enthusiasts are not that far away from what you seem to be missing. @MoritzLost have you ever heard someone saying "ProcessWire is great for small websites, but not for larger, more complex ones"? We all know how wrong that is, don't we? That's the way I feel when people here on the forum are talking about missing migration / version control features... PS: I tried to keep this post short and not so emotional. I think I failed, sorry ?

Yes <?php $rm->migrate([ 'templates' => [ 'car' => [ 'fields' => [ 'title' => [ 'label' => 'Name of the Car', 'columnWidth'=>50, ], 'km' => ['columnWidth'=>50], ], ], ], ]); voila. fields "title" and "km" will be 50% width in the car template context. the title field will have a custom label on the car template. It's really that simple most of the time! Change some properties, commit, push, done! What are you missing? Depends. Both are possible. RockMigrations is really also great if you add it later on a project that did not handle migrations before. It really does not care. For example you could add this in your ready.php: <?php $rm = $modules->get('RockMigrations'); $rm->createField('test', [ 'type' => 'text', 'label' => 'Testing RockMigrations', ]); // add field test to home template after the title field $rm->addFieldToTemplate("test", "home", "title"); You'll have your test field right after the title field of the home template. You can do the same for everything you add to your site. Doing it like this, of course, the migration will run on every page request. The great thing about the migrations is that this will just work! It's not ideal of course, but I just want to show how it works. Add another field? Here we go: <?php $rm = $modules->get('RockMigrations'); $rm->createField('test', [ 'type' => 'text', 'label' => 'Testing RockMigrations', ]); $rm->createField('test2', [ 'type' => 'text', 'label' => 'Testing RockMigrations again', ]); $rm->addFieldToTemplate("test", "home", "title"); $rm->addFieldToTemplate("test2", "home", "test"); Now we got two fields. Ok, we want those fields in a row? Lets refactor this (it would be possible using the syntax from above as well, but I just prefer the array syntax as it has everything in place): <?php $rm = $modules->get('RockMigrations'); $rm->migrate([ 'fields' => [ 'test' => [ 'type' => 'text', 'label' => 'Testing RockMigrations', ], 'test2' => [ 'type' => 'text', 'label' => 'Testing RockMigrations again', ], ], 'templates' => [ 'home' => [ 'fields' => [ 'title' => ['columnWidth'=>33], 'test' => ['columnWidth'=>33], 'test2' => ['columnWidth'=>33], ], ], ], ]); Commit, Push, Done. So do you really think manually creating those fields was quicker? Even if you create those fields remotely first and then pull a DB dump and you have at least semi-automated the process and a dump takes "only 15 seconds or so"... I bet migrations are quicker, more versatile, more powerful. Let's say you want to tidy up your project a little bit. Let's add tags for those two test fields: <?php $rm = $modules->get('RockMigrations'); $rm->migrate([ 'fields' => [ 'test' => [ 'type' => 'text', 'label' => 'Testing RockMigrations', 'tags' => 'RockMigrationsTests', // <----- add this ], 'test2' => [ 'type' => 'text', 'label' => 'Testing RockMigrations again', 'tags' => 'RockMigrationsTests', // <----- add this ], ], 'templates' => [ 'home' => [ 'fields' => [ 'title' => ['columnWidth'=>33], 'test' => ['columnWidth'=>33], 'test2' => ['columnWidth'=>33], ], ], ], ]); Thats it! Both fields live under that new tag: Nice and clean! Ok, now let's remove those changes: The IDE will help you find the right method... So we can do: /** @var RockMigrations $rm */ $rm = $modules->get('RockMigrations'); $rm->deleteFields("tags=RockMigrationsTests"); $rm->setFieldData("title", ["columnWidth"=>100], "home"); How long would it take you to delete those two fields manually? Making sure that all checks are met ("you can't delete this field... it is used by XX templates..." *#*&!), going back to the templates editor, removing the field first, then deleting the field, then going back to the home template and setting the title field back to 100%... I don't know the exact workflow - I haven't done that for a long time now ? Yeah and we are not even talking about doing those changes in a team... Have fun ? Yeah, @adrian, I know Tracy can help. Just trying to show how quick you can get when doing things using RockMigrations. Of course there are some situations where you might not know the properties you need to set. But Tracy can help you. Or you inspect the inputfield and lookup the name in your devtools. The nice side effect: You learn a lot about PW and you will understand it a lot better and improve even more. Well... I don't know what more to say. I feel your pain, but I don't really understand why there are many here "complaining" about those missing features of ProcessWire but nobody seems to be using RockMigrations or seems to have given it an hour or two to simply try it out... ?‍♂️ PS: I know the example runs migrations on every page request, which is not ideal. You can come up with your own strategy to prevent that or you can simply use $rm->fireOnRefresh() so migrations will only fire on modules::refresh I'm maybe similar. There's not a single new project not using RockMigrations and custom PageClasses now. It might be overkill for some, but I think for people that are asking questions that you are asking RockMigrations is just the right tool and you are missing something. That would be a pity. We need more people like you here ? Sometimes life as a developer is hard. But we get paid for that ?? I agree on that. I'm working on it and I wished more were doing so ? Why not? I agree it's not perfect, but I've put hard work into making this possible over the last years. And I think I've come very far. I don't know much other systems though, so I might be on the wrong track. If you know any better systems I'm just as happy to hear about them as @adrian

They had a similar thing I've used in Drupal 7 (back in the day), you would edit the Drupal config in the admin and it would track changes which can then be exported and imported or git'ed up and then executed on your live server either pressing a button or by adding a hook

@adrian Last year we've started using Craft CMS for some new projects, which has full support for version control out of the box. In terms of feature set it's quite similar to ProcessWire – you can define your own fields and entry types (similar to ProcessWire templates) and create custom field layouts. It's not a flat-file system, it does use a database. It also provides an API similar to ProcessWire's $pages to query for entries (pages) or other data. To keep track of schema changes, Craft saves all fields and entry type (and every other setting / config) as YAML files in a config folder. Those can be version controlled, merged and reasoned about easily. You can even read a pull request in Github and see what changes it introduces to field/entry type settings, it's great. It's called project config, here's the documentation. Particularly pleasant is that you can still use the backend to create fields and field layouts – every change in the backend is automatically written to the config folder. So you can just create field layouts in the backend and everything you need to track it in git is already there when you're done. To propagate changes between environments, the CLI has commands to apply the project config to the current environment (database). I was thinking about writing a comparison between ProcessWire and Craft CMS regarding feature set, support for version control and development experiences. There are some good ideas there that ProcessWire could really benefit from. BTW, if you want to track content in git as well as configuration – we also looked at Statamic, which does support flat files and can commit every backend edit to git. Haven't tried it though. @szabesz Not sure if it's that easy, making every aspect of ProcessWire version control friendly would require massive reworks. Including making ProcessWire and it's module installable through Composer and reworking the directory structure to have a better distinction between content and config, as well as dependencies and custom code. This would need to be ProcessWire 4 (at least). Not sure this can be done if ryan isn't looking to put massive amounts of work into this. I don't have the impression that this is the direction ProcessWire is going – though I'm happy to be proven wrong.

The good thing that the most important bits needed by native migration features are there. The bad thing is that it is not completely automated. If you already spotted the missing bits, maybe you (including Horst, since he does something similar) could cooperate with Ryan on implementing the "rest"? I am asking this because I am planning to apply this method as well in the future.

I'd love to know your favorite PW alternative - if something else exists with all the features, flexibility and ease of PW which also has full compatibility with version control, that would be perfect. Surely it would require a flat file database, but even that has its problems when it comes to data querying, doesn't it?

Thanks everyone for the insights! Nice to see all the different approaches, and that I'm not the only one struggling to find a 'perfect' solution … @bernhard Thanks for this. Look real close to a version-controllable schema config. Can the array syntax for fields/templates handle field overrides in a template context as well? What about settings that are environment-specific, like specific parent pages in page reference fields? How would you approach an existing site that doesn't use migrations – add all the existing fields to one big migration file? Or include a database dump as the base state and perform all further changes in the form of migrations? You're right, doing everything manually is a pain. But handling everything in code is a pain as well. The problem for me is that too many areas of ProcessWire are not built with version control in mind. Migrations solve field/template configuration. But then you need to keep track of installed modules, if you want to install modules with Composer you need custom config so they're installed in the right folder. Of course you can't put the entire modules folder in gitignore, since site-specific modules go into the same folder. And what about ProcessWire itself, I don't want the entire core in my version control. So again, custom download script. The list goes on. You don't want to keep track of uploaded files, so the asset folder goes into gitignore. But I do want translations in my version control, turns out those are in random directories in the assets folder … the list goes on. There are just so many small issues if you want a 'complete' version controlled ProcessWire site, which to me means (1) the site can be rebuilt completely from the repository alone and (2) there's no external code or dependencies. And if you solve all of these with scripts, those become hard to manage and even harder to enforce in a team … not sure where I'm going with this to be honest ? I'm a bit of an all-or-nothing person, if I can't have 'clean' version control I'd rather not have version control at all. Probably unwise. To be honest, for new projects I'm currently settled on option 3: to not use ProcessWire at all, unless the project won't require any significant updates after launch. Even all the problems mentioned above can be solved one way or another, it's just not worth it compared to other systems where I get full compatibility with version control by default. I don't think this will change as long as ProcessWire doesn't support version control natively. @horst Yeah, that's close to what I'm doing for updates as well. As long as you're keeping track of any field/template changes diligently, it's not too much work. Though it still bugs me doing the same thing twice. And of course it doesn't solve the problem of working on an update with multiple people at the same time. @szabesz We do that for significant updates to older projects where we don't have development versions any more. Though it means the client can't edit anything while updates are being done, and it doesn't work for sites with user-generated content or user accounts where you can't just freeze the live database for a larger amount of time … @elabx So all your sites are identical in feature set? That definitely doesn't apply to our projects ? The point regarding bitbucket pipelines is interesting. Though I'm hesitant to use automated deployments if I have too write too much custom logic for it or mix partially automated deployments with manual steps. Too error-prone and difficult to enforce in a team, in particular if you want to work on site updates with multiple people …

This week there have been various small core updates and fixes but the biggest change was to the installer in 3.0.191. Last week we removed all but the site-blank profile from the core, cutting in half the size of the core. And because of this, the installer now needed to provide more direction about downloading and installing other site profiles. So it now does that and it also links to a new page that describes all of the past and current site profiles, along with additional details on how to create and install site profiles. Speaking of creating site profiles, the Profile Exporter module was also updated this week. It is now PW 3.x exclusive and is updated to recognize and work with various aspects and $config properties that are specific to 3.x. ProcessWire 3.0.191+ and the Profile Exporter module now support a custom /site/install/finish.php file which is a template file that is called when installation of a new ProcessWire and site profile has finished, but before it has cleaned up the installer. It is a plain PHP file that has access to PW's API and the installer, so it can do pretty much anything you could do from a regular template file or module. I added this because I noticed a few issue reports for the Profile Exporter module were requesting support of one thing or another, and I found that nearly all of them could be added just by having a profile-specific finishing file, for those that want it. So if you want your site profile to perform any other types of customizations on top of what you can already do with a site profile, this is the place to do it. This is where things are at this week but perhaps we'll continue to go further with the installer in supporting more things too in the new year, as there have been several good ideas. Thanks for reading and I hope that you all have a Merry Christmas and/or Happy Holidays!

Real programmers use butterflies To everyone using git for automated deployments: How are you handling field/schema changes in between environments? And what about module updates/new modules, module configuration, custom module data (like Hanna Code shortcodes)? And everything else that's in the database but affects the CMS, like selecting a specific page as allowed parent in a Page Reference field? A database dump is of course an option, but once a site is live you can't just import the dump in production since it will overwrite the live contents. Are you writing migrations using one of the migration modules by hand? For every single field, field setting etc? Assuming you're able to handle all the edge cases (mentioned above and otherwise) with custom migration code, isn't that hugely time-consuming? Particularly when it comes to creating complicated field setups like Repeater Matrix fields with multiple types, where some settings are stored separately from the field settings and not all properties are documented or type-hinted … Something else? Anyone got a setup that allows them to use git and deploy from dev to staging/production without manual steps? If so, how are you handling the above issues?

Happy New Year! Hope that you all have a great end to 2021 and start of 2022. No major core updates to report this week, just a few minor issue fixing commits, so no version bump today. The dev branch is getting close to 100 commits (and at 7 versions) above the master branch, and with even more improvements/fixes/optimizations than that. So we may try to get a new master/main version out in early 2022, as I'd really like to get more master versions out in 2022 than we did in 2021. Some portion of our audience does not use the dev branch where most of the activity happens, and so it might be nice to share more of that activity on the master/main branch. That's one of many things I'm thinking about as the New Year approaches and am certain 2022 is going to be a great year for ProcessWire and the community. Hope that you have a great weekend and Happy New Year!

Ok, so I decided to play around with writing a DOMDocument based approach which hooks into Page::render and replaces all: <img src="/site/assets/files/1234/image.png" alt="my image" class="my_image_classes"> with: <picture class="my_image_classes"> <source srcset="/site/assets/files/1234/image.png.webp" type="image/webp"> <img src="/site/assets/files/1234/image.png" alt="my image" class="my_image_classes"> </picture> Because it happens on page render, this approach works with images embedded into RTE fields, as well as those added via regular <img> tags in template files. It automatically copies classes from the original <img> tag to the picture tag. I still don't know whether this is the best approach or not, but so far so good ?