Leaderboard

Continuing from last week's post and discussion, ProcessWire 3.0.218 decouples the modules system from the cache system. Now the modules system maintains its own internal caches (at least once you do a Modules > Refresh). It'll still use the $cache API as a backup (temporarily), but now you can safely export the database without the "caches" table, or even delete the "caches" table, if you want to. It'll get re-created as needed. In this version, work also continued on the new WireCacheInterface (and major updates in WireCache) so that we could support external modules to handle cache storage. This capability is kind of similar to how we support 3rd party WireMail and WireSessionHandler modules. The first example is WireCacheDatabase, which is the default cache storage handler for the core. And today we have a new module called WireCacheFilesystem that replaces the default WireCache database storage with a file-system based storage, once installed. It's not yet clear if there are major benefits one way or the other (cache in database vs. file system), as I've not been able to put all this new code through performance testing yet. I'd definitely be interested to hear if anyone has a chance to test things out. I expect the file system might be faster for reading caches, while the database may be faster for writing caches. At least that's what I found with a few preliminary experiments, but they haven't been very thorough, so take that with a grain of salt. I thought we needed at least 2 examples of classes implementing WireCacheInterface before we'd be ready to support potential 3rd party WireCache modules. I imagine that 3rd party modules getting into dedicated cache options independent of database or file system is where we'll start to see major performance benefits. At least for sites that use the cache heavily. That's all for this week, have a great weekend!

Let me know if you need those changed. https://github.com/search?q=repo%3Aadrianbj%2FTracyDebugger expireNever&type=code

About that: I'm itching (as far as time allows) to turn CacheRedis into a WireCacheInterface compatible module, but there might be a few stumbling blocks. I see that your modules support searching by expiry, which is not something most key-value-stores are designed for since they take care of expiry on their own. It's okay and possible to read the expiration value for an individual key, though, so I'm not completely sure yet how much of an impact a lack of that is going to have, and which applications besides maintenance are affected. More of a headache will be handling expiration constants like WireCache::expireNever, WireCache::expireReserved etc. since they hold dates in the past, meaning entries with those would expire the moment they are written. My gut feeling tells me that a lot more refactoring in WireCache will be necessary to decouple most of the expiration logic from the cacher implementation, but it might be possible to handle most cases by intercepting those values and applying a different logic. Not sure if I'll find time this (long, here in Germany) weekend to take a closer look. Have a great weekend as well!

@adrian I don't think there's any need to change it, as cache handlers that don't support past dates can substitute a future date, i.e. if($expires === WireCache::expireNever) { $expires = wireDate(WireCache::dateFormat, '+1 YEAR'); }

@millipedia I’d love to see your TextformatterLiteVideoEmbed have its own thread in the Forum. Maybe you an kick it off?

@BitPoet This is for the maintenance processes. WireCache performs maintenance every 10 mins and whenever any page or template is saved. The page/template save maintenance only happens if rows have expiry dates prior to WireCache::expireNever. Finding by expiry dates can be optional if the cache (Redis) handles expiration on its own. I'm also not sure how much need for these two constants anymore since the modules are now decoupled from the cache. Though I think TracyDebugger uses expireNever. You can just replace WireCache::expireNever with some date far in the future (a year?). The WireCache::expireReserved can be treated the same as expireNever. It was primarily for the modules-system and I will likely deprecate this option since it's no longer needed. Its only purpose is to behave like expireNever but also prevent the rows from being deleted when a "delete all" action is being performed. We don't need that anymore. There are a few expirations prior to expireNever that could also considered. Either that, or the WireCacheInterface module could just indicate it doesn't support them. Since these are all dates before WireCache::expireNever, they can be easily filtered in the same way. If the cache can't search by expiry, then implementing them would mean mapping to some other property that is searchable by the cache handler. Any WireCacheInterface module doesn't need to know about any of these since all the logic for finding them is mapped to expiration dates: WireCache::expireSave - This expiration means that the cache row should be deleted whenever any Page or Template is saved. WireCache::expireSelector - This is like expireSave above except that it indicates the cache value is JSON-encoded along with a property named 'selector' and this contains a page-finding selector. WireCache loads them all and when the saved page matches any selector then the cache row is deleted. WireCache maps template IDs to dates by doing $expire=date('Y-m-d H:i:s', $template->id); and that can be reversed with $templateID=strtotime($expire); When a page having that template ID is saved, the cache row having that templateID-based expire date is deleted. Like with the above, the WireCacheInterface module doesn't need to know anything about this other than finding rows having those expiration dates, and presumably those could just be mapped to something else. I'm not sure it's necessary to support these other than mapping expireNever to a future date, as I think very few actually use these features. From what you've described, Redis handles a lot of the stuff that WireCache does on its own. If the cache handler (Redis) can handle these tasks on its own, that's great. If I'm understanding what you've described correctly, here's how I would handle it: When find($options) receives a request containing $options['names'] to find, then ignore anything in the $options['expires'], since Redis is never going to return an expired row. When find($options) receives a request with empty $options['names'], and a populated $options['expires'], just return a blank array (no results). When save() receives an $expire date of expireNever or expireReserved, substitute some far future date instead. When save() receives an $expire date prior to expireReserved, throw an Exception to say the feature isn't supported, or substitute 1 hour, 1 day, or whatever you think is appropriate. Once it works with those, if you wanted to support the full feature set, then you could always go back and see about mapping expiration dates less than or equal to WireCache::expireNever to some other searchable property (and using a far future expiration date). But very few people actually use the page/template/selector clearing features (as far as I know), and the core doesn't need them, so they could be considered optional.

Ahh! Reading this in the early morning laying in bed. Almost always start the weekend with PW blog. What I read makes me smile. A great weekend indeed! Thanks!