Weekly update – 11 July 2024

[ryan]

Thank you for all the feedback, I'm enjoying reading it and it's opening up a lot of good ideas, so please keep it coming. I also have some ideas to add. 

With regard to big/major features, I'd ideally like to focus on things that absolutely must be in the core, and not on things that can be built and supported by others as modules. My wishlist would be that ProcessWire focuses more on its roots, being the best it can be as a framework and CMS for running modules. If I can focus on that part of the core, and let others develop major new features with modules, then not only would the core be that much better, but I think I'd have the bandwidth to do things like develop a more formal testing framework, make the docs that much better, have more time for issues and requests, etc.

Things like media/asset managers and web/block builders/editors (as examples) are best built by those that use them. When you are both the user and developer, you can develop and support something better than if you hired someone to develop it. @bernhard is a great example, he has stepped up and built answers to several things that people ask for, and he's both developer and user of those things.

I understand some of these external modules cost money. But when something doesn't cost money to you, it has still cost money and time resources from someone else, the developer, or whoever sponsored the developer. So it's a question of who pays for it and how they are doing so. 

My strategy here has been to focus on developing things that I can use in my projects and very often the clients help cover the cost. That's how much of ProcessWire has been built. Other features that I use but could not afford to develop and support for free have been developed into Pro modules. This helps fund the core too. I don't make enough to survive on that alone, so I do a lot of client work as well, and I like the diversity of work.

When I say that one particular feature is not likely to be developed in PW's core in the next version, I'm not saying that it's a bad idea at all, nor am I rejecting it, but more likely that I don't have the ability to fund that feature at the moment. (But maybe will later). Or in many other cases, it may be that a particular feature might only be useful in some installations rather than most, and thus belongs as an external module.

If something like RockPageBuilder or PageGRID or others could suit your needs on a particular project, support the developer and get it. The more we can do this, the more that developer can do, the more decentralized the responsibility for ProcessWire can be, and the more I think it benefits not just that module and developer, but the entire ProcessWire project. Maybe the developers of these modules would be open to having a free version with less features so people could get to know them better and upgrade when they need it. Kind of like how there is Lister and ListerPro. 

One idea I'd like feedback on. What if ProcessWire 4.x would be just the core and only the modules essential for the "blank" profile? And that's all the core would be. Then, on the download page (or maybe the installer), you could select all the modules you want to be part of your installation. For instance, you know you'll need repeaters, so you check the box for the Repeater Fieldtype. You know you want Tracy, so you check the box for that. At this stage you could select both 1st party and 3rd party modules to basically configure your own ideal version. From here we could also highlight other options like the Page Builders mentioned. There would have to be some predefined options, like a "well appointed" installation that is the same as the core as it is now. It would be a whole different way of building ProcessWire, but much more back to the roots of PW, becoming even more of a community project, sharing the responsibility a little further and getting more people involved, growing the family.

[Mont]

10 minutes ago, ryan said:

One idea I'd like feedback on. What if ProcessWire 4.x would be just the core and only the modules essential for the "blank" profile? And that's all the core would be. Then, on the download page (or maybe the installer), you could select all the modules you want to be part of your installation. For instance, you know you'll need repeaters, so you check the box for the Repeater Fieldtype. You know you want Tracy, so you check the box for that. At this stage you could select both 1st party and 3rd party modules to basically configure your own ideal version. From here we could also highlight other options like the Page Builders mentioned. There would have to be some predefined options, like a "well appointed" installation that is the same as the core as it is now. It would be a whole different way of building ProcessWire, but much more back to the roots of PW, becoming even more of a community project, sharing the responsibility a little further and getting more people involved, growing the family.

Yes please.

[ryan]

Quote

field type "date multiple"
I guess that's a pretty specific one, but would be really helpful to me so i share it anyways.

FYI the ProFields Table field does this quite well, and I'm currently using it for this. You could also do the same with a repeater, but Table would do it more efficiently. If you need 2 dates (like a range) then also note there is FieldtypeDateRange/InputfieldDateRange available in ProFields as well.

[HMCB]

I am probably the least capable person in the community. I was able to build a very nice delivery app two years ago with Processwire. I was proud of it although it needed more polish. Because of being a relative newbie and designer by trade, I built a front-end that was meant to replicate a backend. Why? Because I wasn’t versed enough on creating custom admin pages in PW plus I didn't want it to look like the PW backend. That left me open to security holes, no doubt. But the end result was decently slick. But guess what really bothered me about the PW backend? I didn’t want my client seeing the word “Pages” throughout the UI they’d be working on. PW is powerful and I didn’t want them thinking this app was a glorified website. “Pages” stuck out like a sore thumb. Other CMSs forgo that which makes building something on top of them seem more custom and crafted to the specific use case. For instance, Craft CMS (see attachments) has a modern backend UI that would feel at home to me and yet there is no mention of “Pages.” I think that extends the use of their product beyond just a website. I know some of you may think I’m crazy but it’s been touched upon throughout this thread in one way or another.

PW is the perfect ground floor. Being so opinionated on Pages just seems to pigeon-hole it and especially if the backend UI improved to feel more modern, then we’d have a tool that would allow relative beginners (to custom app development) to build some powerful stuff.

Thanks for listening.

 

Edited July 22 by HMCB

[MrSnoozles]

12 hours ago, HMCB said:

Being so opinionated on Pages just seems to pigeon-hole it and especially if the backend UI improved to feel more modern, then we’d have a tool that would allow relative beginners (to custom app development) to build some powerful stuff.

Agree with that. I really like the Craft 5 backend, from your screenshot at least. Would be nice to have a page tree in the sidebar though for quick navigation when it's used as a cms.
When used as a custom app, what we usually want to do, is to hide everything that says "page" and provide custom views to the user. E.g. we want sections that show "events" or "skyscrapers" in a list/table, the way users know it from other apps.

16 hours ago, ryan said:

ProcessWire fully supports utf8mb4. Many (most?) of us is have used exclusively on new installations for years. 

I think what was meant is to drop utf8 and myisam or at least do not offer those options in the installer. If somone knows what they are doing and they offer advantages it's possible to change it in the database. My guess would be that the majority of people do not know what's the difference and want utf8mb4+innodb. I didn't know back then and was wondering why emojis were not shown on my site. Took a while to find out that I need utf8mb4 instead of utf8.

 

16 hours ago, ryan said:

I think this[composer] might turn off as many people as it turns on. I know I wouldn't like it

Any particular reason why you wouldn't like it?

Edited July 23 by MrSnoozles

[d'Hinnisdaël]

Thanks @ryan for taking the time and getting back to everybody's suggestions. It's great to see all the ideas and inputs in this thread.

Quote

I think [a full composer installation] might turn off as many people as it turns on. I know I wouldn't like it.

There's not much to like or dislike about how composer installs packages, it just happens to be how PHP projects pull in third-party packages these days. This is as much about making newcomers feel at home as it is about making the life of long-time devs easier.

Quote

ProcessWire already supports WEBP and has for awhile. I was working on core AVIF support, but Robin S. released an AVIF module so I put mine on the back burner. And this may be the sort of thing that is better as a module, or maybe it'll be better in the core at some point, I'm not sure. 

Right, WEBP output has been supported for a while. Upload is still limited to JPG/PNG/GIF/SVG, somewhat arbitrarily. JPEG is a lossy format, so why not allow WEBP source images as well — most images on the web today are WEBP, and browsers and operating systems have broad support.

Extending the availabe output formats could be as simple as $image->format('webp')->width(200)->url.

Quote

Right now we don't build core versions in a way that breaks things from one to another. At least not in a way that is known at the time of assigning a version number. When that changes, I think we'll want to adopt semantic versioning, and maybe that's a part of 4.x. Maybe it would benefit 3rd party modules more in the shorter term, so it's worth looking at there first. 

Semantic versioning is about unintended breaking changes as well — introducing new features brings with it the possibility of introducing new bugs, so even the difference between minor version (features) and patch version (fixes) is meaningful when deciding whether to upgrade a live site or not. That part (fixes vs. features) is probably known when bumping the version.

Quote

Good to hear. We do have the PageAutosave module which does Live Preview even better than ProDrafts. This module is available in ProDevTools or ProDrafts boards, but maybe it can have a wider release. 

Interesting, I'll need to check that one out, then. Sounds promising!

[HMCB]

1 hour ago, MrSnoozles said:

Agree with that. I really like the Craft 5 backend, from your screenshot at least. Would be nice to have a page tree in the sidebar though for quick navigation when it's used as a cms.
When used as a custom app, what we usually want to do, is to hide everything that says "page" and provide custom views to the user. E.g. we want sections that show "events" or "skyscrapers" in a table, the way users know it from other apps 

Very well put. Thank you.

[jploch]

18 hours ago, ryan said:

One idea I'd like feedback on. What if ProcessWire 4.x would be just the core and only the modules essential for the "blank" profile? And that's all the core would be. Then, on the download page (or maybe the installer), you could select all the modules you want to be part of your installation.

@ryan I like the idea. This could speed up the initial setup and if done well would also work for non technical users (They never heard of composer). It could be like a modules manager/installer/picker that lists all modules with short description. Like a more compact list/grid of modules (maybe an expandable accordion with checkboxes for each module). Anything that makes it easier to find and install modules from the backend/installer would be a good addition in my opinion.

18 hours ago, ryan said:

If something like RockPageBuilder or PageGRID or others could suit your needs on a particular project, support the developer and get it. The more we can do this, the more that developer can do, the more decentralized the responsibility for ProcessWire can be, and the more I think it benefits not just that module and developer, but the entire ProcessWire project. Maybe the developers of these modules would be open to having a free version with less features so people could get to know them better and upgrade when they need it. Kind of like how there is Lister and ListerPro. 

I have to say doing commercial modules in PW is hard. The target group is small and the marketing of the module is the sole responsibility of the developer (eg. building a website/shop to promote and get the module). It would be nice if there was a platform integrated into the PW site to list/promote and maybe even buy commercial modules (Not only Ryans pro modules but also commercial modules build by the community). What was asked here shows the interest in page/block builders, and I think both RockPageBuilder and PAGEGRID are already very promising and have the potential to attract more users to PW (Which in return would make it easier for other developers to sell modules). I could envision a light/free version of PAGEGRID, but I need to justify the time to maintain and develop it further, so I think there needs to be a commercial aspect to drive the project forward.

[szabesz]

@Ray Thanks for taking the time to write your explanations and examples posted here.

Currently, I am more interested in "multi-site" than "multi-instance", but supporting multi-instance can also be essential for sure. By "multi-site", I mean that the project I am working on requires multiple frontends, managed and served by a single instance of ProcessWire, providing a "unified admin" in that manner.

As I need to further develop this site this year, I will take your advice and utilize delegation the way you explained. Thanks!

[szabesz]

20 hours ago, ryan said:

Quote

The current versioning chaos is a drawback for sure.

For real? This is the first I've heard of any chaos.

If my wording was too harsh, I apologize for that.

What I meant was that the fact that integers and strings can be used for module version numbers causes unnecessary issues on its own, for example:

https://processwire.com/talk/topic/13389-adminonsteroids/page/39/

Quote:

issue was: Any one else having trouble with this module not detecting updates with Ryan's ProcessWireUpgrade module? Some of my sites are stuck at AOS version 1.6.71 and it but it also reports 1.6.7.2. As you can see in the picture, there is another version available but it doesn't give me an option to download/update it.

reply was: Sorry, that's my fault, I've mixed the string vs numeric versioning and that's why the chaos (adrian reminded me). Next time I'll double check :)

comment was: Also, next time for string versions, I think you should leave off the trailing period :)

Also, if one reads the whole thread of @netcarver's ModuleReleaseNotes Module, then there is a good view there to see how many variables he had to tackle: https://processwire.com/talk/topic/17767-module-release-notes/

 

I think sticking to semantic versioning would solve most of these issues and would also support this:

19 hours ago, ryan said:

What if ProcessWire 4.x would be just the core and only the modules essential for the "blank" profile? And that's all the core would be. Then, on the download page (or maybe the installer), you could select all the modules you want to be part of your installation.

Since if there are no strict versioning requirements, then it is hard to imagine a robust and extensible module manager in the first place, no to mention Composer support.

 

[szabesz]

Regarding page content creation by "site editors" in the admin, since TinyMCE's future regarding its open-source state is questionable, Dan Brown at BookStack started to implement a replacement based on lexical:

Building a custom editor would also allow ProcessWire to cater to its own needs, like a live preview feature described as:

On 7/20/2024 at 11:30 PM, d'Hinnisdaël said:

The editing experiencing itself doesn't need to resemble the viewing experience — most of our clients are perfectly happy with a well-implemented split-screen, where they edit the page blocks on the left, and get a live preview on the right.

It would be nice to be able to implement sort of block editor-like features in the RTE (and share them with the community).

More complex block editors have their use cases and clients for sure, but for most projects, being able to do some sort of simple block editing would be enough, just like Ryan's example here: https://processwire.com/blog/posts/using-tinymce-6-in-processwire/#example-of-using-extra-css-styles. But, instead of very limited techniques like that, a custom RTE could provide a more robust solution without being a full-blown frontend block editor like RockPageBuilder and PageGRID. Such a custom RTE would automatically be supported by Ryan's PageAutosave module and its Live Preview feature.

 

 

[szabesz]

BTW, BookStack should be considered as the next generation for ProcessWire's documentation system, I think.

Edited July 23 by szabesz
typo

[teppo]

22 hours ago, ryan said:

One idea I'd like feedback on. What if ProcessWire 4.x would be just the core and only the modules essential for the "blank" profile? And that's all the core would be. Then, on the download page (or maybe the installer), you could select all the modules you want to be part of your installation.

Please no ?

To elaborate, there are a couple of issues with this:

• Depending on how this is implemented, it could be a major turn-off for users like me, who have never (and likely will never) click that "download" link on the website, let alone use a configuration tool to get a download package. Most of the time I'm installing ProcessWire via Composer or by cloning it via Git, and obviously my wish would be that this process isn't made more complicated.
• On the other hand I've seen some users (beginners) kind of trip on things like "you'll have to install a module to get Repeaters working" or "you need to get a separate site profile". As such, I don't think this would necessarily be a good idea considering new users either. A configuration tool would likely be interesting mostly for advanced users, who know exactly what they want... and prefer to use a download link on a website to get it, which (I would assume) somewhat limits its usefulness.

That being said, as long as I can still get a "viable" version of ProcessWire (most of what's currently in the core, and future additions as well, assuming that they are meaningful ones — which they undoubtedly will be!) without having to use a configuration tool, I of course don't mind if it is added as an additional option. If someone likes that kind of thing, then great, why not!

Also if this would mean that non-essential core modules could be also installed one by one via Composer, that would be nice ?

What I'm missing here is the context: is this about getting a smaller downloadable package? Is that still a thing, e.g. is it really a problem for many users? Serious question, because I really don't know. I care a lot about performance on websites, but have never considered the download size of a CMS I'm using; unless it's in gigabyte range or something crazy like that. Or is it about dropping support for some core features? I'm not getting that vibe from here either, but that would make more sense to me, though in that case it might be better to just drop them from the core and perhaps find new maintainers for the packages if possible ?

Or is there some other benefit I'm missing?

[d'Hinnisdaël]

20 minutes ago, teppo said:

Quote

One idea I'd like feedback on. What if ProcessWire 4.x would be just the core and only the modules essential for the "blank" profile? And that's all the core would be. Then, on the download page (or maybe the installer), you could select all the modules you want to be part of your installation.

Please no ?

I'll agree with this — a modular download is not required and would create more confusion than it helps clear. Newcomers don't know which modules they need ("what's a repeater?") and probably don't want to wade through a list of modules before trying out ProcessWire. Experienced devs tend to use most core features anyway, or work from their custom boilerplate repo. So I'm failing to see the benefit here, as file size shouldn't be of concern on a web server.

[interrobang]

My main wish for ProcessWire 4 would be a generally more modern admin theme (e.g. with Ajax-Save unpoly/htmx style) and that at least fields that are also used in the frontend (FormBuilder) no longer have a jQuery dependency. 

On the subject of jQuery, it would also be great if the updates announced last year could also be activated for live pages in the next major release.

[Jonathan Lahijani]

With regards to SEO related features, I think that being natively built in to ProcessWire is not the best choice as it goes against the unopinionated nature of ProcessWire.  If one were to develop a webapp where SEO is not necessary, those fields would cause bloat.  But if this were to be an official Profield, it would be wonderful.

Personally, I use the FieldsetGroup Pro field (I gave SEOMaestro a try but it didn't work the way I liked), but since that's now legacy, I will have to switch my approach at some point in the future.  It looks like this (note: it used to have XML sitemap fields as well, however when WireSitemapXML came out, I removed them and let that module handle it):

If there were to be an new SEO module similar to my screenshot (meaning it supports fields for X, Facebook and other social networks), I would hope it's not officially called "FieldtypeSEO" but rather something like "FieldtypePresence" (ChatGPT helped me with that one) since it's more fitting (it helps improve your site's "presence" via search engines and social media networks).  Personally I dislike the word SEO as it represents a lot of what's wrong with the internet and the snake-oil behind it.

[Laksone]

7 hours ago, Jonathan Lahijani said:

With regards to SEO related features, I think that being natively built in to ProcessWire is not the best choice as it goes against the unopinionated nature of ProcessWire.  If one were to develop a webapp where SEO is not necessary, those fields would cause bloat.

I agree...

The way I handle it is that I have made a script that creates all the necessary fields (title, meta description, OG and so on...) via API if I need the SEO features in the project. I do this once when creating the project. There might be a better way, but this actually works pretty well for me.

Edited July 23 by Laksone

[Jonathan Lahijani]

On 7/22/2024 at 6:58 AM, ryan said:

But if you find it beneficial to use /site/classes/YourPageClass.php then also consider a separate class (ProductPageTools.php?).

@ryan Assuming one were to take this approach, what is the proper/recommended way to load the ProductPageTools.php class file?

[MrSnoozles]

Again, regarding a more modern administration: I like how the Silverstripe backend works, both how the page tree is presented, as well how the page edit screen looks. I also like that expanding the tree works in a (imo) more standard way (meaning a click will edit the page, to expand it there's an arrow). I think we could get some inspiration from there. You can try the demo here, no need to sign up: https://demo.silverstripe.org/admin/

 

[bernhard]

Hey @ryan first of all let me thank you for ProcessWire in general and for all the work you put into it! Also thank you for mentioning me and my modules. Unfortunately I can second what @jploch said, that it's very hard to offer commercial modules in PW, but that's a topic for another post as this one has gotten very long already ? 

NOTE: Sorry, this post got very long, but I think it covers some important concepts and I think some of the concepts can really push PW forward on the web-application front. I think it's important to understand the details, so I ask you kindly to read carefully and try to understand the bigger picture ? 

In this post I'd ask for clarification about what you said about custom page classes attaching their own hooks:

On 7/22/2024 at 3:58 PM, ryan said:

Quote

I do not think anyone suggested having "more" initialization methods. Instead, it would be nice to see fewer of them while still being able to initialize everything necessary. 

I think the phrase "everything necessary" might be worth considering here. A class ideally just has one responsibility. Page instances are there to provide an interface to one page's content and be swapped in and out of memory on demand. When there are things like hooks going into a Page, that changes the role completely. If you start doing this a lot on a large site then it may affect scalability. Maybe it's fine for one individual installation or another, however. 

I didn't want to believe that, because everything that I've built over the last few years relies heavily on the concept of "MagicPages" which is a feature of RockMigrations and which basically turns every custom page class into an autoload module where you can add hooks that are related to that pageclass. This concept makes maintainability of the whole project a lot easier imho. Before I invented that concept I had quite complex applications and they had hundreds of lines of code in /site/ready.php or /site/init.php;

---------- first some background, later the tests ------------------

Spoiler

The next step was to have a /site/init.php like this:

<?php
include 'init-foo.php';
include 'init-bar.php';
include 'init.baz.php';

This was ok, but in all those files you are still in a procedural context and you don't have any of the benefits that OOP has to offer. That's why MagicPages were born. With Magicpages you can turn this (without namespaces to make it easier to read):

<?php
// site/init.php
$wire->addHookAfter('ProcessPageEdit::buildForm', function(HookEvent $event) {
  $p = $event->process->getPage();
  if($p->template == 'foo') return;
  $form = $event->return;
  $form->add(...);
});

// site/classes/FooPage.php
class FooPage extends Page {
}

Into that:

<?php
// site/classes/FooPage.php
class FooPage extends Page {
  use MagicPage;
  
  public function editForm($form) {
    $form->add(...);
  }
}

What is important to understand here is the following: "editForm" is a method that is only executed for FooPage! You don't have to add checks like if($page->template == 'whatever') .. and so on. The idea is that the pageclass is solely responsible for things related to that page object.

How does that work?

See https://github.com/baumrock/RockMigrations/blob/ab2c3232f0de8994ca08c0a76a5de89a74240abe/MagicPages.module.php#L83-L91

• RockMigrations builds an array of all templates using the "MagicPage" trait on every modules refresh.
• For every magicpage template it will create one instance of that page at runtime.
• If that page has an init() method, it will call that method.
• If that page has a ready() method, it will add the page instance to the "readyClasses" array and call that method at a later stag (when pw triggers ready() of MagicPages.module.php)
• If not disabled via settings RM will attach magic methods like "editForm" shown above which is similar to getPageListLabel() and provides an easy interface for some commonly needed hooks that I think are best placed in the page class itself.

What this also means is that I don't think that this adds a noticable overhead and should impact scalability. We only ever load one instance of that magicpage per template, so there might be 10 or 20 more pages in memory, but not hundreds or thousands. We only every trigger init() and ready() ONCE per request, not for every instance!

I hope that makes sense. I know that you are not happy with getPageListLabel() as it is related to the admin and not only frontend stuff. I was quite surprised when I first heard that. So I'd be very interested in details why you think a pageclass should only be related to frontend stuff and not to backend things. I also ask you to maybe throw that concept/mindset aboard as I think it makes perfect sense to put both frontend and backend related things into those page classes.

In my daily work the process is always the same: I need a helper method for my $page object, like $page->renderHumanDate(...) or whatever. So I go to /site/classes/EventPage.php and add the method "renderHumanDate()" to that class. I need to modify the backend behaviour of EventPage objects? Go to /site/classes/EventPage.php and add "getPageListLabel()" or "editForm()" or whatever.

Another important thing that I like to put in those pageclasses is migrations. Please have a look at this realworld pageclass, which is responsible for handling events of that installation:

<?php

namespace ProcessWire;

use RockMigrations\MagicPage;

class EventPage extends DefaultPage
{
  use MagicPage;

  const tpl = 'event';
  const prefix = 'event_';

  public function init(): void
  {
    $rm = rockmigrations();
    $rm->setPageNameFromTitle($this);
  }

  public function date(): string
  {
    return 'tbd: return a human readable date';
  }

  public function migrate()
  {
    $rm = $this->rockmigrations();
    $rm->migrate([
      'fields' => [],
      'templates' => [
        self::tpl => [
          'fields' => [
            'title' => [
              'label' => 'Name des Events',
            ],
            Site::field_daterange,
            RockPageBuilder::field_blocks,
          ],
          'parentTemplates' => ['events'],
        ],
      ],
    ]);
  }
}

I'll explain everything step by step:

• First, we define the class and extend the DefaultPage (great feature, thx for that!)
• Then we use the MagicPage trait which will add support for init() where we can attach hooks and which will also trigger migrate() whenever the file is changed
• Then we add the "tpl" constant which is necessary for MagicPages and we add a "prefix" which is just used to prevent naming conflicts of fields
• Then we add the init() method which is triggered ONCE on every request. This is the place where I attach all necessary hooks.
• In init() we load RockMigrations and we use the "setPageNameFromTitle()" helper. This will make sure that the page name is updated when the page title is changed. It will also lock this field and show a note to the user:
  
• Then we add a method that will output the event date in a human readable way, eg "1. - 3. Feb. 2024"
• Then we add the migrate() method which will create all fields necessary for "Event" pages:
  • we don't add any fields (yet), we only add existing fields
  • for the template we set some fields
    • we reuse the global title field and set a custom label
    • then we add the field "Site::field_daterange" which is created by the Site.module.php module so that it can be used in several templates
    • then we add the RockPageBuilder blocks-field, which is created by the module RockPageBuilder and can also be used on any template
  • then we set the allowed parents of "event" pages to the "events" template. In the events template we set "childTemplates" to "event" accordingly.

Why is that so great? Because you can develop locally extremely fast and once you are done push everything to production in a fully automated way and all fields and changes will be there instantly, see this screencast. Note that the window on the right reloads automatically whenever I save the file on the left!

I also rely heavily on that concept for RockPageBuilder's "block" pages and I got a lot of positive feedback how well everything is structured and how great it is to work with, so I'm not really happy if you @ryan advice not doing this. Maybe you could clarify please now that you have more context. Thx!

And finally here are the tests that I ran to see whether MagicPages or larger pageclass files have any noticable impact on performance. Spoiler: I didn't observe one ? 

----------  the tests ------------------

Spoiler

Ok so I did some tests to get real numbers instead of just talking and guessing...

First, I created a test template (in Site.module.php's migrate() method):

// test memory usage
$rm->createTemplate('test');
$rm->createPage(
    template: 'basic-page',
    parent: 1,
    name: 'tests',
);

Then I created a simple RockShell command to create 10k pages:

<?php

namespace Site;

use ProcessWire\Page;
use RockShell\Command;

class SiteCreatepages extends Command
{
  public function handle()
  {
    $parent = $this->wire()->pages->get('name=tests');
    for ($i = 1; $i <= 10000; $i++) {
      $p = new Page();
      $p->template = 'test';
      $p->parent = $parent;
      $p->title = "Test $i";
      $p->save();
      $this->write("Created Page {$p->title}");
    }
    return self::SUCCESS;
  }
}

And around 10s later we have 10.000 pages to test:

Then I created another RockShell command to test memory usage:

<?php

namespace Site;

use RockShell\Command;

use function ProcessWire\wireBytesStr;

class SiteMemoryusage extends Command
{
  public function handle()
  {
    $this->write(wireBytesStr(memory_get_usage(true)));
    return self::SUCCESS;
  }
}

Which returned this:

Quote

➜  git:(main) ✗ rockshell site:memoryusage
8,0 MB

Next Test:

Quote

// added this line
$this->wire()->pages->find('template=test');

// result
38,0 MB

Next Test:

Quote

// changed find() to findMany()
// result
24,0 MB

Next Test:

  public function handle()
  {
    $all = $this->wire()->pages->find('template=test');
    foreach ($all as $p) $this->write($p->title);
    $this->write('MEM: ' . wireBytesStr(memory_get_usage(true)) . ', Peak: ' . wireBytesStr(memory_get_peak_usage(true)));
    return self::SUCCESS;
  }

// result
Test 9996
Test 9997
Test 9998
Test 9999
Test 10000
MEM: 38,0 MB, Peak: 38,0 MB

Next, I created a custom pageclass for the template "test" and ran the test again:

<?php

namespace ProcessWire;

class TestPage extends Page
{
}

Still 38MB

I confirmed that the class is working via Tracy:

Next test: Turn the page into a MagicPage:

<?php

namespace ProcessWire;

use RockMigrations\MagicPage;

class TestPage extends Page
{
  use MagicPage;
}

Still 38MB

Next test: Add a hook via init() which is automatically triggered by RockMigrations and modifies the page edit screen for test pages.

<?php

namespace ProcessWire;

use RockMigrations\MagicPage;

class TestPage extends Page
{
  use MagicPage;

  public function init(): void
  {
    wire()->addHookAfter('ProcessPageEdit::buildForm', $this, 'hookBuildForm');
  }

  protected function hookBuildForm(HookEvent $event): void
  {
    $form = $event->return;
    $page = $event->process->getPage();
    if (!$page instanceof self) return;
    $form->insertAfter([
      'type' => 'markup',
      'label' => 'Testing Memory Usage',
      'value' => 'MagicPages are great :)',
    ], $form->get('title'));
  }
}

Quote

Test 9998
Test 9999
Test 10000
MEM: 38,0 MB, Peak: 38,0 MB

Next I used the tracy console to create 1000 methods for the TestPage pageclass:

I copied that over to VSCode/Cursor and used multicursor to create 1000 fake methods for that class:

Guess what? 38MB? Almost ? 

Quote

Test 9997
Test 9998
Test 9999
Test 10000
MEM: 38,0 MB, Peak: 40,0 MB

And the TestPage.php pageclass has more than 5000 lines of code and is 69kB "large":

The next one was interesting:

<?php

namespace Site;

use RockShell\Command;

use function ProcessWire\wireBytesStr;

class SiteMemoryusage extends Command
{
  public function handle()
  {
    $p = $this->wire()->pages->get('template=test');
    while ($p->id) {
      $this->write($p->title);
      $p = $p->next;
    }
    $this->write('MEM: ' . wireBytesStr(memory_get_usage(true)) . ', Peak: ' . wireBytesStr(memory_get_peak_usage(true)));
    return self::SUCCESS;
  }
}

Quote

Test 9997
Test 9998
Test 9999
Test 10000
MEM: 178,0 MB, Peak: 178,0 MB

Next I added $pages->uncacheAll() in the while loop...

Quote

Test 9998
Test 9999
Test 10000
MEM: 152,0 MB, Peak: 152,0 MB

Interesting that with a $pages->find() and foreach() all 10k lines where printed almost instantly whereas when using while() it took around 30s.

Then I ran the same test as above after removing all the fooXX() methods from the pageclass:

Quote

Test 9998
Test 9999
Test 10000
MEM: 152,0 MB, Peak: 152,0 MB

Then I removed the MagicPage trait from the class to get a clean and empty custom pageclass again... still 152MB!

Then I created another 10k pages and ran the memory test again with the empty pageclass (and with uncacheAll()).

Quote

Created Page Test 2 X 9996
Created Page Test 2 X 9997
Created Page Test 2 X 9998
Created Page Test 2 X 9999
Created Page Test 2 X 10000

// the command
rockshell site:memoryusage

// result
Test 2 X 9998
Test 2 X 9999
Test 2 X 10000
MEM: 296,0 MB, Peak: 296,0 MB

Next I tried 20k pages again with the foreach() approach:

Quote

// using find()
MEM: 64,0 MB, Peak: 68,0 MB

// using findMany()
MEM: 38,0 MB, Peak: 38,0 MB

Another approach - this had a huge impact:

  public function handle()
  {
    $done = false;
    $found = [];
    while (!$done) {
      $p = $this->wire()->pages->get([
        'template' => 'test',
        'id!=' => $found,
      ]);
      if (!$p->id) $done = true;
      else {
        $found[] = $p->id;
        $this->write($p->title . ' - ' . wireBytesStr(memory_get_usage(true)));
      }
      $this->wire()->pages->uncacheAll();
    }
    $this->write('MEM: ' . wireBytesStr(memory_get_usage(true)) . ', Peak: ' . wireBytesStr(memory_get_peak_usage(true)));
    return self::SUCCESS;
  }

Quote

Test 4627 - 392,0 MB
Test 4628 - 392,0 MB
Test 4629 - 394,0 MB
Test 4630 - 394,0 MB
Test 4631 - 394,0 MB
Test 4632 - 394,0 MB
Test 4633 - 394,0 MB
Test 4634 - 394,0 MB
Test 4635 - 394,0 MB
Test 4636 - 394,0 MB
Test 4637 - 394,0 MB
Test 4638 - 394,0 MB
Test 4639 - 394,0 MB
Test 4640 - 394,0 MB
Test 4641 - 396,0 MB
Test 4642 - 396,0 MB
Test 4643 - 396,0 MB
Test 4644 - 396,0 MB

I stopped execution as it took quite long and the memory usage obviously grew by 2MB every some pages.

I thought maybe it's related to the $found array or to the $p->next call in the previous runs. But it doesn't look like that:

  public function handle()
  {
    while (true) {
      $p = $this->wire()->pages->get('template=test');
      if (!$p->id) return self::SUCCESS;
      $p->delete();
      $this->write("Deleted page #$p - " . wireBytesStr(memory_get_usage(true)));
      $this->wire()->pages->uncacheAll();
    }
  }

Again it grew 2MB every some pages: Deleted page #21110 - 266,0 MB

----------------

Very, very long story short: @ryan I'd appreciate if you could explain in more detail why you think it is bad to put - what I'd call - pageclass-related code into pageclasses? Also I'd like to invite you to re-think if it is really bad to have things like "getPageListLabel" in the pageclass. I think this is a great to have these methods and I really enjoy having "editForm($form)" as a shortcut for ProcessPageEdit::buildForm if($page->template == ...) and so on. At the very minimum I'd love to have an init() and a ready() method where we can add hooks as we need them.

Thank you very much for reading. I'm sorry for the length, but I hope there was maybe something interesting in it.

Please let me know if I did anything wrong with my tests or what I could do better. Thx.

Long live ProcessWire ? 

[ryan]

@bernhard

Good questions and I'll answer according to the design of the core. These considerations are rigid when it comes to what the core does with these Page classes, but flexible outside of that. You are using Page classes in a way that gives them multiple roles. Whether that's good or not depends on your architectural point of view. But it sounds like you are happy with how it's working, and you've got a well defined strategy, and this is valuable. So none of this may useful for your case, so keep that in mind. 

Quote

We only ever load one instance of that magicpage per template, so there might be 10 or 20 more pages in memory, but not hundreds or thousands. We only every trigger init() and ready() ONCE per request, not for every instance!

That sounds okay to me. Why does it load an extra 10-20 pages? Or does that just depend on what pages are used in the request? As long as that magic page interface isn't increasing resource usage each individual Page object significantly, then it should scale just fine. 

Quote

So I'd be very interested in details why you think a pageclass should only be related to frontend stuff and not to backend things. 

It shouldn't be related to either. It should be independent of environment. A page is a type that aggregates content for a location in the tree. The problem with having front-end-only or admin-only stuff in a Page object is that it's code in the page that steps outside of its role and applies for one environment and not another. It can be seen as a type of bloat, turning a Page object into a jack of multiple trades.

If you've got significant code that's just for one environment or another then the idea is that it's better to keep it separate from the Page. Perhaps delegated to a different class like my earlier example (or even a function outside of a class). But if it's just for a one-off case or something then I wouldn't bother. 

Take that getPageListLabel() method, that's something to represent the Page in the admin only, which is why it makes me uneasy. It's a tradeoff. It's a one-off case, so I think it's okay as long as we don't add more like it. Potentially the method might have front-end value in some cases too, so maybe that's another reason it's an okay compromise. But if we found ourselves starting to put a lot of admin-specific methods in Page classes then those would be better in a separate supporting class or function, or better yet, decouple it from the class entirely. That way it doesn't add code to the page class for something that goes outside its responsibility. 

Another point is that often code added for one environment or another will be the same between different page classes. To avoid repeating yourself, you'd have to repeat the same methods in multiple classes, or more likely add complexity with more levels of inheritance or traits to share that code. Moving that code to a dedicated class makes it more contextual, maintainable and extendable across multiple classes. 

Quote

The idea is that the pageclass is solely responsible for things related to that page object.

There's that word "things" which seems like it might mean "everything" in this context. Here we're treating the Page like it's a Swiss Army knife for the type, rather than the type itself. It's like being the car and the carwash, the lawnmower and the grass, or not just the ball but the bat and glove too. That editForm() method looks like it's got the Page class now involved in manipulating forms in the admin. In that case Page is not just a type representing an aggregate of content, but also a form editor, adding application logic, giving the Page class multiple unrelated responsibilities. Some responsibilities which apply to all instances of the page and others which might only apply to one instance on occasion. 

If the Page must be the gatekeeper for multiple responsibilities, the idea is that it would be preferable that it delegates those responsibilities rather than implements them itself. That would enable it to provide that interface without expanding the internal role of the class. 

If you find benefits in using them differently then I'm not suggesting you change anything. Since you asked I have to give you an accurate answer of what Page classes are originally designed for and what I consider best practices around them. These are how the core considers them and doing likewise ensures things stay simple, efficient and scalable in the long term. But in PW flexibility is more important than rules and this is also true of Page classes. There's more than one way to do things well, I like your creativity and if you've found something that works well for your projects over time and scale then stick with it. 

[cwsoft]

6 hours ago, bernhard said:

At the very minimum I'd love to have an init() and a ready() method where we can add hooks as we need them.

@ryanCould you comment on this quote from @bernhard in this thread. Hope I haven‘t missed it in this long thread. Having init and ready methods for page classes is something which was asked or proposed quite often in this forum or discussed at PWs Github repo in the past. Would be great to hear your thoughts on that and if this is something you may consider for future PW versions. Thanks in advance.

[ryan]

@cwsoft  These methods don't make sense in the core, unless they are to initialize each individual instance of a Page class. But Page classes (like all objects in PW) already have methods for that: __construct() and wired(). As I understand the methods requested here, they are independent of Page instance, so honestly don't belong in a core Page class, and are not something that I think the core should suggest. See what I wrote above for why. Though they are trivial to implement on your own with a static variable should you want it. I'd still advise against it, especially for hooks, but here's an example:

class MyPage extends Page {
  static private $ready = false;
  public function wired() {
    parent::wired();
    if(!self::$ready) {
      // your init code... 
      self::$ready = true;
    }
  }
}

 

[Jonathan Lahijani]

@ryan In my original reply to this post I mentioned making PW a bit stronger for web application development.  After reading these replies, it seems that a lot of the more intricate techniques that would be desired for advanced use cases are already possible natively, but even those of us here that I would consider highly experienced with PW and OOP are still picking up a few crucial nuggets of information that go a long way in structuring a web application with PW (at least it seems that way).

So to elaborate on what I stated originally, I think demonstrating how you would approach some of these advanced techniques natively in PW (showing that a feature request to make it "nicer" is not necessary / technically problematic) in a real working app would be very helpful.  I believe you attempted this with the site-invoices profile and building it out further with more web-application-y type techniques would really be great.

[FireWire]

On 7/23/2024 at 1:27 AM, jploch said:

I have to say doing commercial modules in PW is hard. The target group is small and the marketing of the module is the sole responsibility of the developer (eg. building a website/shop to promote and get the module). It would be nice if there was a platform integrated into the PW site to list/promote and maybe even buy commercial modules (Not only Ryans pro modules but also commercial modules build by the community). What was asked here shows the interest in page/block builders, and I think both RockPageBuilder and PAGEGRID are already very promising and have the potential to attract more users to PW

While this isn't a suggestion for the core, since it came up I want to second this. There are many great modules that would benefit from this- for both those that are currently available, and just as importantly, modules that have yet to be built. I can say for myself that there are some things that I'd love to create but it's difficult to justify considering the effort to build, maintain, and improve it without the potential to recoup some of the real costs involved.

Listing the benefits of an official shop would make this comment far too long. I'm a big fan of the official Pro modules and I don't build sites without them, same has become true for those offered by @bernhard. This would also really help newcomers adopt ProcessWire if the modules directory could become a platform to browse both paid and free modules. Decoupling the Pro module purchase experience from the forums would be really helpful and I would argue foster wider adoption. I see this as a long-term sustainability initiative for the ecosystem as a whole.

Maybe this could tie into ProcessWire core (hear me out)

Maybe this could be supported in ProcessWire itself where the module is downloaded in ProcessWire (as free modules are currently), and then keys that are purchased in the store can be entered where modules are managed in the admin itself. Reducing the effort and friction of adopting high quality modules for more projects and providing true integrated support. I would love to install Pro modules as easily as those in the directory.

I would expect, and hope, to see a per-sale fee for commercial modules sold in an official shop to both make this endeavor sustainable and to see something go back to the ProcessWire project. By that I mean I would like to build modules that buy @ryan a beer, preferably multiple because that means things are going well.

There have been some great suggestions here, but I do want to take a moment for some thanks to the constant improvements that I've seen. Even the smaller quality of life changes in new versions are entirely appreciated as we enjoy a better developer experience. Cheers! ?