Weekly update – 8 December 2023

[ryan]

This week I've continued work on the page versions support that I wrote about last week. While the main PagesVersions module needs more work before it's ready to commit to the dev branch and test externally, it is so far working quite well in internal testing. 

I mentioned last week how it will support an interface where Fieldtypes can declare that they will handle their own versions. That interface has been pushed to the dev branch as FieldtypeDoesVersions. I've also implemented it with the Repeater Fieldtype, which is visible starting here. Repeaters are so far working really well with versions!

As far as core Fieldtypes go, Repeater, PageTable and FieldsetPage are likely the only ones that will need custom implementations. For ProFields, RepeaterMatrix already works with the implementation in the core Repeater (already tested). It's possible that other ProFields will not need custom implementations, though not yet positive.

The module that provides version capability is called PagesVersions and the plan so far is to make it a core module that provides API version support for pages. A separate module provides interactive version support in the page editor. I've built this module initially so that I can test versions more easily, but it'll be expanded to provide a lot more. Below is a screenshot if what it looks like in the page editor Settings tab so far:

As you can see, you can Edit or Restore any existing version. You can also create a new version from the Live version, or any other version. And of course you can view and delete any versions. When you Restore a version, it essentially replaces the live version with the one that you are restoring.

All of this can be done from the module's API as well. Note that the API is provided by a $pagesVersions API variable that is present when PagesVersions module is installed. The API method names and such are a bit verbose right now but may be simplified before it's final. 

// Get page and add a new version of it
$page = $pages->get(1234);
$page->title = 'New title';
$version = $pagesVersions->addPageVersion($page);
echo $version; // i.e. "2"

// Get version 2 of a page
$pageV2 = $pagesVersions->getPageVersion($page, 2);

// Update a version of a page
$pageV2->title = "Updated title";
$pagesVersions->savePageVersion($pageV2); 

// Restore version to live page
$pagesVersions->restorePageVersion($pageV2);

// Delete page version
$pagesVersions->deletePageVersion($pageV2);

Thanks for reading! More next week. 

 

[David Karich]

Hey @ryan, I'm so excited about the feature. The two big things I think ProcessWire is missing are versioning and a lack of global asset management. ProDrafts was a good approach so far, but had limitations. Can you perhaps briefly explain how the versioning module works? ProDraft had the approach that everything I save is saved in a draft until I publish. Is this also the approach here, that every time I save a page a new version of the last state before saving is created or do I have to create a new version myself beforehand, in which I work, revise a draft and then "restore" it when I'm done as a form of "fake" publishing?

[MrSnoozles]

@David KarichVery good points. I'm very excited for this feature too. IMO the things we and our clients have been missing the most in ProcessWire, apart from versioning, are (in that order):
- global asset management / media manager with directories to reuse assets in various places. The lack of this is is often the sole reason we can not recommend ProcessWire for some projects
- automatic backend translations using ProcessWire in any language other than English is suboptimal, because usually the backends will end up being half in <target language>, half in English
- a proper multisite solution to handle multiple domains / channels in a single instance. The Multisite module is a good start, but having more isolated channels and sharing/isolation options for each channel would make this a much better fit
 

[ryan]

@David Karich ProDrafts doesn't have versioning per se, it just maintains a draft version. So the version support will be something different, though the plan is that ProDrafts will be able to eventually use it for maintaining its draft versions. As for how the versioning works, it's very simple in that you have API methods to add, get, save, and delete versions of a page. That's the extent of it. While technically you could code a hook to make it save a new version on every page save, I don't think it would be practical, as a version is an entire copy of a page, including its files. Most likely one would use it to create a version intentionally, and perhaps work on it independently of the live version of the page. Or maybe they would use it to create a restorable snapshot or backup of a page's current state, before making some other changes. I think there are a lot of possibilities for how someone might use it, but for now I'm just focused on its API and making it all work reliably across different fieldtypes. 

@MrSnoozles A media manager is something I've not needed here, and I get off track and mess things up when I start developing stuff tools I don't have a use for in my own projects. Nearly everything that's in ProcessWire's core is stuff that I develop and maintain because I use it in my client work. When I need to share file or image assets across multiple pages I build a page or group of pages for the purpose, and that is ideal for my projects. A big part of the reason why PW as a framework is so modular and hookable is so that there is always the opportunity extend it in ways that are useful for your own projects.  For example, @kongondo built what looks to be a really cool media manager for ProcessWire, though I don't have experience with it beyond reading about it and watching the videos.

As for translations of the admin, PW's admin is already 100% translatable, so if you are seeing a mixture of English and another language, then that means that the language pack you have installed is missing translations. But that's something you'd need to ask the author of the language pack about. You can also always submit a pull request to the translation author's language pack, or create your own. PW makes this all very simple. 

Multisite is another example of a module that I just have not yet had a need surface for in my own work. Though I'm interested in it, so always looking for the opportunity. I'd have to come up with a case where I would use it and find it worthwhile. Otherwise I don't think I could do as a good job of a job as someone that is actively using it. I think the existing multi-site module(s) were built by someone (Soma, Apeisa?) that was using them in their projects. Worth mentioning too that PW will happily serve multi-site already, so long as you add all the hostnames to $config->httpHosts and use conditionals like if($config->httpHost === 'host.domain.com') to decide what content to serve. What I find useful in my work here is multi-instance, being able to boot other sites when and where needed, like we do on this site, where the modules directory actually comes from a separate ProcessWire installation, despite being delivered by the main site. 

[MrSnoozles]

@ryan Good points. The slim core and the enormous flexibility is why so many love ProcessWire and trust you with your decisions. Still I want to comment on one point:

42 minutes ago, ryan said:

As for translations of the admin, PW's admin is already 100% translatable, so if you are seeing a mixture of English and another language, then that means that the language pack you have installed is missing translations. But that's something you'd need to ask the author of the language pack about. You can also always submit a pull request to the translation author's language pack, or create your own. PW makes this all very simple. 

I know it is simple, but the reality is that in client projects it often still ends up being a mix of <language> and English, because not all modules provide translations (the community seldom submits pull requests for languages) and there is no time to translate everything. I think that ProcessWire is lacking in this area has to do with you rarely needing it in another language but English.
My idea how this could be improved would involve three relative simple steps:

1. Set up a community translation tool, like Weblate (https://weblate.org/), where people that want to contribute can easily see which translations are still missing. Weblate is really good feature-wise, but the interface for adding translations could be simpler, so another tool might fit better.
2. Write a crawler that goes through all the uploaded modules and core files, extract the translatable phrases and upload those packages to Weblate
3. Write a ProcessWire module that maps the languages defined in ProcessWire with country codes, then automatically downloads the matching translation files from Weblate whenever a module is added or updated

 

In general: would that something you would consider useful? If so I could try implementing a POC when I find the time.

 

[kongondo]

On 12/11/2023 at 3:32 PM, ryan said:

Worth mentioning too that PW will happily serve multi-site already, so long as you add all the hostnames to $config->httpHosts and use conditionals like if($config->httpHost === 'host.domain.com') to decide what content to serve.

I didn't know this! Where does this code go, ready.php or init.php? Or in config.php?

[bernhard]

On 12/11/2023 at 4:53 PM, MrSnoozles said:

In general: would that something you would consider useful? If so I could try implementing a POC when I find the time.

Have you seen RockLanguage? There was no feedback at all...

[szabesz]

1 hour ago, kongondo said:

Where does this code go, ready.php or init.php? Or in config.php?

As far as I understand, what Ryan suggests is putting if($config->httpHost === 'host.domain.com') and such into template files.

Instead of putting if's into template files for deciding what to render based on the current host, I do something like (and more...): if (file_exists($hostTemplatePath)) $templates->get($templateName)->filename = $hostTemplatePath; in init.php to decide what file to use as a template in the first place. In the very same loop where I set the actual template, I also do if (class_exists($namespace . $className)) call_user_func_array("{$namespace}{$className}::initiate", []); so my page classes can get initiated upon their request.

[monollonom]

On 12/11/2023 at 4:53 PM, MrSnoozles said:

In general: would that something you would consider useful? If so I could try implementing a POC when I find the time.

Personally I don’t really see how setting something like this would entice the community to contribute more than making a pull request to a module’s repository to share your translation in your language (https://processwire.com/blog/posts/pw-3.0.181-hello/).

I’m mostly working on english websites (or for people who are ok with an english admin) but if one day I’m required to use french and I need to translate some modules then I’d happily share my work this way.

And if you need a shortcut, you can always give a shot at @FireWire ’s excellent Fluency translation module 🙂

[MrSnoozles]

14 minutes ago, monollonom said:

Personally I don’t really see how setting something like this would entice the community to contribute more than making a pull request

You could be right on that. I'm thinking however that it would be beneficial for three (subjective) reasons:

1. It's easier to see the status of translations and contribute to them from a GUI.
2. I personally don't want to create a PR for just some of the translations. With that system you could add some suggestions here and there
3. It would be easier for non-tech people, although admittedly most of the ProcessWire users probably are

 

4 hours ago, bernhard said:

Have you seen RockLanguage? There was no feedback at all...

Haven"t seen it, but it does seem a lot easier than the core way to add translations.

[FireWire]

6 hours ago, monollonom said:

And if you need a shortcut, you can always give a shot at @FireWire ’s excellent Fluency translation module 🙂

Well, since @monollonom tagged me I thought I'd tinker. I added a one-click button to Fluency that translates any module file, core or otherwise, in seconds. It's integrated with the native ProcessWire language translator pages. Some details here. It makes it entirely possible for everything a user interacts with in the admin to be translated within minutes, not hours.

10 hours ago, bernhard said:

Have you seen RockLanguage? There was no feedback at all...

I didn't know about this module... this looks great!

On 12/11/2023 at 3:53 PM, MrSnoozles said:

I know it is simple, but the reality is that in client projects it often still ends up being a mix of <language> and English, because not all modules provide translations (the community seldom submits pull requests for languages) and there is no time to translate everything. I think that ProcessWire is lacking in this area has to do with you rarely needing it in another language but English.

Language is a tough nut to crack and applying a universal solutions are difficult. It's huge task for a module developer, contributor, or a core maintainer like Ryan, to take on when they are providing free and open source software for the community. The fact that ProcessWire handles language so well to begin with shows a thoughtful approach by Ryan and ProcessWire contributors, especially if it is as rarely as you assume. Try Fluency and let me know if it helps out.

[FireWire]

On 12/11/2023 at 2:27 AM, MrSnoozles said:

The lack of this is is often the sole reason we can not recommend ProcessWire for some projects

Can you share what platform(s) you use as an alternate to ProcessWire in these instances? As the developer of a language module maybe there is some inspiration to be had.

[MrSnoozles]

19 hours ago, FireWire said:

Can you share what platform(s) you use as an alternate to ProcessWire in these instances?

That was related to the media manager. Typo3 for example offers good media manager capabilities. https://demo.typo3.org/typo3 (just choose any username to start the demo)

[cb2004]

Could we see a version 3.1 in the dev branch this week? I am always wondering what it would take to get that digit rising.

[teppo]

On 12/11/2023 at 5:53 PM, MrSnoozles said:

In general: would that something you would consider useful? If so I could try implementing a POC when I find the time.

Parts of it, at least, yes!

I've had for a very long time on my todo list a related idea; basically something like the https://translate.wordpress.org/ portal for WordPress. Though I didn't honestly even consider using pre-built solutions.

I don't say this lightly, but this is an area where WordPress is way ahead ProcessWire, in my opinion. Just a few reasons why I think their system is so good:

• It's a shared, easy to use, web-based, non-developer friendly system. Whether I'm a contributor or someone wanting to install a translation for a plugin, I don't need to figure out how author of plugin x wants to handle their translations, and I also don't need to know or figure out how to create a GitHub account, open pull requests, etc. This point alone is a major benefit and lowers the bar for contributing 🙂
  • Just for the record, many plugins still bundle translations with the source code. So yes, both systems can co-exist.
• Anyone can contribute without core/theme/plugin author(s) having to do anything. I for one have access to Finnish translations for a few plugins. Permissions are handed down by general translation editors for the locale, which is much easier than having to contact individual plugin authors who may or may not be around and active, and may or may not be interested in bundling some language they have no personal interest in.
• Once a locale has enough coverage, it automatically becomes installable via the admin and via WP-CLI. Personally I use WP-CLI for managing site translations, as it makes things a lot easier to automate. Installing and updating translations is literally one command away, which — as someone who manages quite a few sites — is a big bonus.
  • This, of course, only works for plugins that have been translated this way. Not all have. It's not a perfect world.
• Finally, the system handles versions properly. For an example if I wanted to submit a correction for the Finnish translations for ProcessWire version 3.0.184, there's no easy way to do that. Maintainer of the Finnish translations repository could set up separate branches or some other way to handle it, but this would be a case by case thing, and there's no guarantee that the language pack maintainer is interested in such a thing.

In the past I've also contributed via Launchpad, which (at least back then) was essentially the same thing for free / open source software, and worked really well in my opinion.

Anyway, I'm still interested in this idea, but have not had time to pursue it myself. Maybe one day, unless someone else solves it properly before me 😉

In my opinion it is a viable idea, but there are many questions to answer, and quite a few of those are not straightforward. Also I believe that in order to gain traction this would need to be somehow integrated into our current translation tool(s) and it shouldn't be too opinionated (for an example I personally would prefer it to not update anything completely automatically, as was suggested here earlier.)

[MrSnoozles]

On 12/15/2023 at 8:22 PM, teppo said:

I've had for a very long time on my todo list a related idea; basically something like the https://translate.wordpress.org/ portal for WordPress.

Yes, that's exactly what I had in mind too. Wordpress really drives me insane as a developer, but the fact that almost all the backend is in the clients language is one of the few things I envy when using ProcessWire.

The advantage is that everyone can submit translations in a matter of seconds, when you're bored or just waiting for your train. No need to clone a repo, go through the source files and submit a PR.
Core: https://translate.wordpress.org/locale/de/default/wp/
Plugins: https://translate.wordpress.org/locale/de/default/wp-plugins/