Weekly update – 24 June 2022: Simple list cache

[ryan]

This week core updates are focused on resolving issue reports. Nearly all of the 10 commits this week resolve one issue or another. Though all are minor, so I'm not bumping the version number just yet, as I'd like to get a little more in the core before bumping the version to 3.0.202.

This week I've also continued development on this WP-to-PW site conversion. On this site hundreds of pages are used to represent certain types of vacations, each with a lot of details and fields. Several pages in the site let you list, search and filter these things. When rendering a list of these (which a lot of pages do), it might list 10, 20, 100 or more of them at once on a page (which is to say, there can be a few, or there can be a lot). Each of the items has a lot of markup, compiled from about a dozen fields in each list item. They are kind of expensive to render in terms of time, so caching comes to mind. 

These pages aren't good candidates for full-page caches (like ProCache, etc.) since they will typically be unique according to a user's query and filters. So using the $cache API var seems like an obvious choice (or MarkupCache). But I didn't really want to spawn a new DB query for each item (as there might be hundreds), plus I had a specific need for when the cache should be reset — I needed it to re-create the cache for each rendered item whenever the cache for it was older than the last modified date of the page it represents. There's a really simple way to do this and it makes a huge difference in performance (for this case at least). 

Here's a quick recipe for how to make this sort of rendering very fast. But first, let's take a look at the uncached version:

// find items matching query
$items = $pages->find('...'); 

// iterate and render each item
foreach($items as $item) {

  echo "
    <!-- expensive to render markup here --->
    <div class='item'>
      <a href='$item->url'>$item->title</a>
      ...and so on...
    </div>
  ";
}

That looks simple, but what you don't see is everything that goes in the <div class="item">...</div> which is a lot more than what you see here. (If we were to take the above code literally, just outputting url and title, then there would be little point in caching.) But within each .item element more than a dozen fields are being accessed and output, including images, repeatable items, etc. It takes some time to render. When there's  100 or more of them to render at once, it literally takes 5 seconds. But after adding caching to it, now the same thing takes under 100 milliseconds. Here's the same code as above, but hundreds of times faster, thanks to a little caching:

// determine where we want to store cache files for each list item
$cachePath = $config->paths->cache . 'my-list-items/';
if(!is_dir($cachePath)) wireMkdir($cachePath);

// find items matching query
$items = $pages->find('...'); 

// iterate and render each item
foreach($items as $item) {

  $file = $cachePath . "$item->id.html"; // item's cache file
  if(file_exists($file) && filemtime($file) > $page->modified) {
    // cache is newer than page's last mod time, so use the cache
    echo file_get_contents($file);
    continue;
  }
  
  $out = "
    <!-- expensive to render markup here --->
    <div class='item'>
      <a href='$item->url'>$item->title</a>
      ...and so on...
    </div>
  ";

  echo $out; 

  // save item to cache file so we can use it next time
  file_put_contents($file, $out, LOCK_EX); 
}

This is a kind of cache that rarely needs to be completely cleared because each item in the cache stays consistent with the modification time of the page it represents. But at least during development, we'll need to occasionally clear all of the items in the cache when we make changes to the markup used for each item. So it's good to have a simple option to clear the cache. In this case, I just have it display a "clear cache" link before or after the list, and it only appears to the superuser:

if($user->isSuperuser()) {
  if($input->get('clear')) wireRmdir($cachePath, true); 
  echo "<a href='./?clear=1'>Clear cache</a>";
}

I found this simple cache solution to be very effective and efficient on this site. I'll probably add a file() method to the $cache API var that does the same thing. But as you can see, it's hardly necessary since this sort of cache can be implemented so easily on its own, just using plain PHP file functions. Give it a try sometime if you haven't already. Thanks for reading and have a great weekend!

[nbcommunication]

Just implemented this in combination with a $cache-ing of the $pages->find() query. Lightning fast.

if(file_exists($file) && filemtime($file) > $page->modified) {

Would it not be $item->modified?

Cheers,

Chris

[MarkE]

This looks a nice idea and I was looking to implement something similar on a site of mine. However, many pages have repeater matrix fields. If these are changed directly, rather than through the host page edit, then the modified date for the host page is not changed (even though the updated results are shown). I guess there are two ways of fixing this:

1. hook on saving the repeater item to set the new mod date for the host page
2. check all mod dates - for host and repeater pages - rather than just the host page

Any ideas which approach is more efficient? I guess I'm more inclined to option 1.