ProcessWire 3.0.24 and 2.8.24 enhance page traversal

ProcessWire 3.0.24 and 2.8.24

This week updates were focused on covering GitHub issue reports and feature requests, plus some great new performance improvements to our page traversal methods.

Upgrades to page traversal methods

Every Page object has several traversal methods that provide a simple API for you to traverse around a family of Page objects. This week, the following Page traversal methods got major upgrades, making them a whole lot faster, easier to use, and more efficient:

All of the above methods also accept a $selector string, enabling you to filter what is returned.

Past reasons to avoid these methods

If you've ever used any of these methods, then you know that in the past they've come with a lot of warnings about overhead. It small scale, these methods were super convenient and fast. But if you were dealing with any kind of scale, these methods basically weren't worth using anymore because they were too slow and cumbersome. The reason for that is that no Page has knowledge of where it is relative to its siblings. Pages are sorted by the database when they are retrieved, as a group. So the location of a page relative to its siblings can only be known in that context. Meaning, all the siblings also have to be loaded in order to answer these kinds of traversal questions. That keeps our architecture very flexible, but it's not so convenient when you want to know "what is the next sibling?" and similar questions, at a large scale.

Finding a solution to overhead

This week the time was taken to correct that, something I've been meaning to do for a long time. Actually it's been tried on a few occasions before, but always run into one issue or another that prevents it from being realized. The same thing happened this week again. Two days were spent coming up with a solution that worked in 80% of cases, only to find there was another 20% of cases were it couldn't possibly work. Unfortunately sometimes you just have to work through a problem before you can spot the loose ends. It was back to the drawing board. But too much time had gone into it to set it aside once again. So after another day and a half, now I think we are there with these page traversal methods now working fast and well!

Performance (new vs. old)

Lets take the most basic (and probably most useful) $page->next() method. This method has always been just fine if you weren't dealing with hundreds or thousands of pages. But the bigger the scale, the slower it gets. Calling it on a page with 10000 siblings, the execution time was 12 minutes (yes minutes!). Amazingly enough PW didn't run out of memory and was actually able to load all those 10k pages to figure things out. Now this is on an older slow notebook computer, but 12 minutes is also just insane overhead, we are talking serious bottleneck.

As of ProcessWire 3.0.24 the same exact $page->next() call with 10000 pages now takes about 100 milliseconds (0.1 seconds) on the same slow notebook computer. If children are sorted by a custom field, there's more overhead, but still in the milliseconds range to execute the call. As before, the larger the scale, the slower it will get. But if we're fast performing the calculation among 10000 other pages, then I think we're pretty fast. People generally don't use these traversal methods to traverse millions of pages. And if you are needing to work with this many pages at once via the API, then you should probably be using $pages->findMany() either way.

No longer do you have to worry about potential overhead with these page traversal methods, nor do you need to worry about anything extra like you did before (i.e. pre-filtered siblings or include=mode considerations). Now these methods work just as simply, easily, and reliably as the rest of PW's API methods.

New “Save and edit next” button action in page editor

This one came in as a feature request on GitHub last week, and it's part of what prompted the refactoring of those page traversal methods above. We couldn't possibly implement a "save and edit next" feature without having a $page->next() method that could scale. However, now that we've got it, adding this new button action was a piece of cake. We hope that you find it useful, I know I already have.

New $page->index() API method

This new $page->index() method (also accessible as a property) tells you the position of any page relative to its siblings. It's very simple, just returning an integer with a zero-based index. However, this was information we could not have easily determined before unless in a group of manually sorted pages. It's again the new refactored page traversal methods that made this one possible. When comparing the value returned by this method with the numChildren method/property on the parent, it gives you an accurate picture of where a given page lives relative to its siblings, without those siblings having to be loaded.

That's it for this week! Next week we've got some nice improvements to our ImageSizer classes by way of Horst, plus we've got some nice enhancements to the selector engine that I think you will like! Have a great weekend and be sure to read the ProcessWire Weekly.

Comments

  • matjazp

    matjazp

    • 4 years ago
    • 50
    That's great! What an improvement in processing speed! Would be nice if you could explain how you did it.
    • Teppo

      Teppo

      • 4 years ago
      • 00
      You're not the only one wondering. From what I can tell it seems that until now next() and prev() used to load *all* siblings to memory at once and then iterate until a match was found, while now they only load required items. There's a bit more to it, so hoping Ryan will chime in soon :)
    • ryan

      ryan

      • 4 years ago
      • 00
      More info on that here: https://processwire.com/talk/topic/13694-pw-3024-2824/#comment-123406
  • thetuningspoon

    thetuningspoon

    • 4 years ago
    • 30
    I love that PW is not just adding features, but improving performance under the hood with every new release.
  • Can

    Can

    • 4 years ago
    • 00
    nice :-)

    maybe I missed it, but how could I add custom page save buttons?

  • thetuningspoon

    thetuningspoon

    • 4 years ago
    • 00
    Would this make adding new pages to the top of the list with manual sort a practical possibility now?
  • ottogal

    ottogal

    • 3 years ago
    • 00
    In addition to the very useful “Save and edit next” button action I would like to have a "Close and edit next" button action too.
    Often I am screening some property or setting through many siblings and find me doing these unnessecary Saves just to profit of the helpful "Edit next" action.
 

PrevProcessWire 3.0.23 and 2.8.23

2

This week we have a few nice tweaks for those using multi-language support and translation tools in ProcessWire. Plus improvements to the 2.8.x branch, and a new feature for repeaters. More 

Twitter updates

  • Weekly update for Jan 10, 2020—front-end file uploads in LoginRegisterPro with InputfieldFrontendFile module: More
    10 January 2020
  • If you haven't seen it yet, be sure to check out this great infographic by @teppokoivula in ProcessWire Weekly #294— How ProcessWire professionals work—the results of our recent weekly polls summarised: weekly.pw/issue/294/#how…
    3 January 2020
  • Today we have a new master version released, 3.0.148! There are 25 new versions worth of upgrades, fixes and optimizations in this new master version, relative to the previous. In this post we take a closer look at what’s new, how to upgrade, & more— More
    3 January 2020

Latest news

  • ProcessWire Weekly #297
    In the 297th issue of ProcessWire Weekly we're going to check out the latest core updates (ProcessWire 3.0.149) and introduce a new third party module called StaticWire. Read on!
    Weekly.pw / 18 January 2020
  • ProcessWire 3.0.148 new master
    Today we have a new master version released, version 3.0.148! The last master version was 3.0.123, so there are 25 new versions worth of upgrades, fixes and optimizations in this new master version, relative to the previous. In this post we’ll take a closer look at what’s new, how to upgrade, and more.
    Blog / 3 January 2020
  • Subscribe to weekly ProcessWire news

“ProcessWire is like a breath of fresh air. So powerful yet simple to build with and customise, and web editors love it too.” —Margaret Chatwin, Web developer