ProcessWire 3.0.24 and 2.8.24 enhance page traversal

ProcessWire 3.0.24 and 2.8.24

This week updates were focused on covering GitHub issue reports and feature requests, plus some great new performance improvements to our page traversal methods.

Upgrades to page traversal methods

Every Page object has several traversal methods that provide a simple API for you to traverse around a family of Page objects. This week, the following Page traversal methods got major upgrades, making them a whole lot faster, easier to use, and more efficient:

All of the above methods also accept a $selector string, enabling you to filter what is returned.

Past reasons to avoid these methods

If you've ever used any of these methods, then you know that in the past they've come with a lot of warnings about overhead. It small scale, these methods were super convenient and fast. But if you were dealing with any kind of scale, these methods basically weren't worth using anymore because they were too slow and cumbersome. The reason for that is that no Page has knowledge of where it is relative to its siblings. Pages are sorted by the database when they are retrieved, as a group. So the location of a page relative to its siblings can only be known in that context. Meaning, all the siblings also have to be loaded in order to answer these kinds of traversal questions. That keeps our architecture very flexible, but it's not so convenient when you want to know "what is the next sibling?" and similar questions, at a large scale.

Finding a solution to overhead

This week the time was taken to correct that, something I've been meaning to do for a long time. Actually it's been tried on a few occasions before, but always run into one issue or another that prevents it from being realized. The same thing happened this week again. Two days were spent coming up with a solution that worked in 80% of cases, only to find there was another 20% of cases were it couldn't possibly work. Unfortunately sometimes you just have to work through a problem before you can spot the loose ends. It was back to the drawing board. But too much time had gone into it to set it aside once again. So after another day and a half, now I think we are there with these page traversal methods now working fast and well!

Performance (new vs. old)

Lets take the most basic (and probably most useful) $page->next() method. This method has always been just fine if you weren't dealing with hundreds or thousands of pages. But the bigger the scale, the slower it gets. Calling it on a page with 10000 siblings, the execution time was 12 minutes (yes minutes!). Amazingly enough PW didn't run out of memory and was actually able to load all those 10k pages to figure things out. Now this is on an older slow notebook computer, but 12 minutes is also just insane overhead, we are talking serious bottleneck.

As of ProcessWire 3.0.24 the same exact $page->next() call with 10000 pages now takes about 100 milliseconds (0.1 seconds) on the same slow notebook computer. If children are sorted by a custom field, there's more overhead, but still in the milliseconds range to execute the call. As before, the larger the scale, the slower it will get. But if we're fast performing the calculation among 10000 other pages, then I think we're pretty fast. People generally don't use these traversal methods to traverse millions of pages. And if you are needing to work with this many pages at once via the API, then you should probably be using $pages->findMany() either way.

No longer do you have to worry about potential overhead with these page traversal methods, nor do you need to worry about anything extra like you did before (i.e. pre-filtered siblings or include=mode considerations). Now these methods work just as simply, easily, and reliably as the rest of PW's API methods.

New “Save and edit next” button action in page editor

This one came in as a feature request on GitHub last week, and it's part of what prompted the refactoring of those page traversal methods above. We couldn't possibly implement a "save and edit next" feature without having a $page->next() method that could scale. However, now that we've got it, adding this new button action was a piece of cake. We hope that you find it useful, I know I already have.

New $page->index() API method

This new $page->index() method (also accessible as a property) tells you the position of any page relative to its siblings. It's very simple, just returning an integer with a zero-based index. However, this was information we could not have easily determined before unless in a group of manually sorted pages. It's again the new refactored page traversal methods that made this one possible. When comparing the value returned by this method with the numChildren method/property on the parent, it gives you an accurate picture of where a given page lives relative to its siblings, without those siblings having to be loaded.

That's it for this week! Next week we've got some nice improvements to our ImageSizer classes by way of Horst, plus we've got some nice enhancements to the selector engine that I think you will like! Have a great weekend and be sure to read the ProcessWire Weekly.

Comments

  • matjazp

    matjazp

    • 4 years ago
    • 50
    That's great! What an improvement in processing speed! Would be nice if you could explain how you did it.
    • Teppo

      Teppo

      • 4 years ago
      • 00
      You're not the only one wondering. From what I can tell it seems that until now next() and prev() used to load *all* siblings to memory at once and then iterate until a match was found, while now they only load required items. There's a bit more to it, so hoping Ryan will chime in soon :)
    • ryan

      ryan

      • 4 years ago
      • 00
      More info on that here: https://processwire.com/talk/topic/13694-pw-3024-2824/#comment-123406
  • thetuningspoon

    thetuningspoon

    • 4 years ago
    • 30
    I love that PW is not just adding features, but improving performance under the hood with every new release.
  • Can

    Can

    • 4 years ago
    • 00
    nice :-)

    maybe I missed it, but how could I add custom page save buttons?

  • thetuningspoon

    thetuningspoon

    • 4 years ago
    • 00
    Would this make adding new pages to the top of the list with manual sort a practical possibility now?
  • ottogal

    ottogal

    • 3 years ago
    • 00
    In addition to the very useful “Save and edit next” button action I would like to have a "Close and edit next" button action too.
    Often I am screening some property or setting through many siblings and find me doing these unnessecary Saves just to profit of the helpful "Edit next" action.
 

PrevProcessWire 3.0.23 and 2.8.23

2

This week we have a few nice tweaks for those using multi-language support and translation tools in ProcessWire. Plus improvements to the 2.8.x branch, and a new feature for repeaters. More 

Twitter updates

  • ProcessWire 3.0.163 resolves more than a dozen issue reports and adds new hooks and configurable module install options, among numerous other updates. In addition, a major update to ProCache (v4.0) has been released— More
    24 July 2020
  • Weekly update for 17 July 2020 and a look at new features coming in next week's version of ProCache— More
    17 July 2020
  • ProcessWire 3.0.162 continues improvements to selector operators, fixes various minor issues, adds new API convenience methods, improves documentation, optimizes and refactors various portions of code and DB queries, and more— More
    10 July 2020

Latest news

  • ProcessWire Weekly #325
    In the 325th issue of ProcessWire Weekly we're going to cover a new master version of ProcessWire, introduce the Textformatter Audio Embed module, and more. Read on!
    Weekly.pw / 1 August 2020
  • Powerful new text-searching abilities in 3.0.160
    In ProcessWire 3.0.160 we’ve got some major upgrades and additions to our text-search abilities. This brings a whole new level of power to $pages->find() and similar API calls, especially when it comes to search engine type queries.
    Blog / 19 June 2020
  • Subscribe to weekly ProcessWire news

“We were really happy to build our new portfolio website on ProcessWire! We wanted something that gave us plenty of control on the back-end, without any bloat on the front end - just a nice, easy to access API for all our content that left us free to design and build however we liked.” —Castus, web design agency in Sheffield, UK