PW 3.0.200 – New master/main version

[ryan]

This new main/master version has more than 220 commits, resolves more than 80 issues, adds numerous new features, performance improvements and optimizations, and consumes HALF the disk space of our previous release—

https://processwire.com/blog/posts/pw-3.0.200/

I've just merged the dev branch to the master branch so this new version is available now. I will add the 3.0.200 tag (which should trigger packagist and others) over this weekend or Monday. 

[kongondo]

Congrats @ryan and the whole community!?

[cb2004]

Nice work as always.

[Jan Romero]

These always trip me up, I get all excited by a big blog post and then it turns out I already know everything from following your lovely weekly threads ? Great stuff as always!

[Robin S]

Wow, what an awesome list of improvements!

@ryan, I have a question about the lazy-loading of fields. Do you think this change means that the previous recommendation to reuse fields, as in the "Making efficient use of fields in ProcessWire" blog post, is now no longer needed?

Say I am using text fields in lots of different templates for different purposes, and if I create and name fields so they describe the use in each template I will have 50 text fields, or if named them generically and reused them I would have maybe 5 text fields. Is their any reason now to prefer the 5 fields option over the 50 fields option, given that in any one template I'll only have 5 text fields or less and so presumably only ~5 or less fields get fully loaded at a time?

[bernhard]

Congrats @ryan that's a really great achievement ? 

Also thx to @matjazp for all your help on the github issues - great work!

[kongondo]

It's not often I stumble on parity like this, so grabbed this screenshot for posterity ?.

 

[teppo]

19 hours ago, Robin S said:

@ryan, I have a question about the lazy-loading of fields. Do you think this change means that the previous recommendation to reuse fields, as in the "Making efficient use of fields in ProcessWire" blog post, is now no longer needed?

Also curious about this one. Is lazy-loading enough to make number of fields a non-issue?

Mostly just thinking out loud here, but I do see at least one benefit in sticking with fewer fields: your fields may remain more manageable, especially if the alternative is to have a lot of very specific fields. Consider having to make the same change to settings of 50 fields compared to 5 fields, for an example. (Of course assuming that they are identical enough for this to make sense in the first place.)

On the other hand I guess that under some circumstances more fields could even be good for performance, considering that each field has a separate table. This seems unlikely to cause noticeable difference, though, unless we're talking about an absolutely massive scale ?

[bernhard]

1 hour ago, teppo said:

Mostly just thinking out loud here, but I do see at least one benefit in sticking with fewer fields: your fields may remain more manageable, especially if the alternative is to have a lot of very specific fields.

For me the opposite is true ? The main reason for that might be my workflows around RockMigrations and custom page classes. Having the fields defined in code directly in the page class is so much easier for me than any other approach I tried. So those additions where very welcome ones for me ? Need some new data for a boat? Go to the boat pageclass file and add a field there. Need a new field for every blog post? Go to the blog post pageclass file and add it...

I'm only sharing fields across templates (page classes) when they really have the same purpose (eg a RockMatrix field for the pages content that has several content-blocks like headline, text, quote, etc that should be the same for templates home, basic-page and blog-item). So adding a new content block like "image" for example would make it available on all those templates immediately compared to having to update all templates one by one and adding the new image block to three different fields.

For me it feels a lot better to have a single field for a single purpose. Sharing a field for different purposes by overriding settings in template context never felt good to me... I'm using template context a lot, but only to do simple modifications like tweaking the field's label (eg of the title field).

One huge benefit of such an approach can be that you get reusable components that you can simply copy from one project to another. That's because the page classes do NOT share fields with other components of your site and therefore are self-contained parts that work on their own. That would not be possible if you shared fields from a central place. Or it would be a pain to reuse single parts of that website. I have been there. I don't want back ? 

[ryan]

Quote

I have a question about the lazy-loading of fields. Do you think this change means that the previous recommendation to reuse fields, as in the "Making efficient use of fields in ProcessWire" blog post, is now no longer needed?

@Robin S @bernhard @teppo

I would say it's less important to limit how many fields you create than it was before, but it's still worthwhile to make efficient use of fields for other reasons (at least for the way I work). But the technical reasons for doing so appear to be much less than before.

If you look here at the performance numbers on a system with 1000 fields you can see that prior to lazy loading, it took 0.5167 ms (half second) to load the fields. This might sound reasonable, but note the fields were created with an automated script and not as diverse as they would be in a real environment, so I'd expect the numbers to be higher/slower in real life. After lazy loading that part of the boot process was reduce to 0.0062 ms. Even without lazy loading enabled, the field loading was reduced to 0.0322 ms (in that 1000 field environment) just due to the other optimizations made in support of lazy loading. So there's some question as to whether lazy loading is even necessary anymore now that everything else more optimized. But using less time and energy is always good thing, even if not felt on individual requests, small differences add up to significant numbers over thousands of requests. 

The way lazy loading works is that everything in the database table (for fields, templates, fieldgroups) still gets selected and loaded, but into an array rather than into objects. That data doesn't get loaded into and converted to actual Field (or Template, Fieldgroup) objects until that specific object is requested. TheTuningSpoon found that overhead came from actually creating and loading the objects rather than selecting them from the DB, and I was able to confirm that. But it also means that the more fields you have, the more memory they will consume. But the memory consumed by the raw field data is very little and I think you'd have to have many thousands of them for it to even be a consideration. 

Lazy loading fields don't change the way I use fields, at least. I like to have as few fields as necessary just because it's easier for me to keep track of, fewer things to remember when writing code, fewer things to manage during changes, etc. But if your workflow is like the one Bernhard describes above then that seems like a case where more fields would appear to be a benefit. 

[Sergio]

We upgraded a big multi language site to 3.0.200 and to PHP 8* + MySQL 8 a couple of days ago, several thousands pages, lots of templates and fields (120MB DB).

The upgrade went fine, and the site is running super fast! I didn't measure with tools, but just by reaching the URLs of some complex pages and you can see it loading waaay faster. The client could not be happier.  ?

Fantastic job, Ryan and the other contributors! 

Thank you,

Sergio

---

* I used RectorPHP to help upgrading my custom modules, which was nice. 

[thetuningspoon]

We had an incident where we tried to upgrade one of our sites to 3.0.200 and needed to roll back (we had only intended to upgrade the staging site but some configuration confusion led to the live database also being affected!) The site was previously on 3.0.165. When we tried to roll back the files, we were unable to because it resulted in numerous errors about missing module classes being triggered inside of other modules. Here are some examples:

Uncaught Error: Class 'ProcessWire\InputfieldText' not found in /var/www/live/wire/modules/Inputfield/InputfieldEmail.module:16

Uncaught Error: Class 'ProcessWire\InputfieldCheckbox' not found in /var/www/live/site/modules/InputfieldSlideToggle/InputfieldSlideToggle.module:3

Uncaught TypeError: Argument 1 passed to ProcessWire\Pagefile::__construct() must be an instance of ProcessWire\Pagefiles, null given, called in /var/www/live/site/templates/model/TransmissionProcessor.php on line 598 and defined in /var/www/live/wire/core/Pagefile.php:131

Exception: Method Languages::hasPageNames does not exist or is not callable in this context (in wire/core/Wire.php line 564)

Fatal Error: Uncaught Error: Class 'Fieldtype' not found in site/modules/FieldtypeDecimal/FieldtypeDecimal.module:13

This persisted even after clearing compiled files and triggering a module refresh. Can anyone think of what database changes might have been triggered by the upgrade from 3.0.165 to 3.0.200 that would result in these sorts of errors? I have occasionally "downgraded" ProcessWire before without having this kind of issue, or at least it wouldn't persist after refreshing modules and clearing compiled files.

[Migu]

Have just ugraded to the latest version with PHP 8.16! Went everything smoothly ! Thanks a lot Ryan and the other contributors ! PW is the best CMS !!!