Over 100k rows in WireCache table

[Zeka]

Hi.

I will have more than 100k in the cache table. Most of theme are updated on a daily basis.  Could there be some unobvious pitfalls of such intensive usage (at least for me)? 

Thanks, Eugene. 

[BitPoet]

You might reach a point where table level locks kill (some of) the speed improvements of caching if you are using MyISAM as the database engine. In that case, switching to InnoDB might speed things up, though you'll lose a small bit of that in insert/update performance itself.

I have a site with about the same number of cache entries that runs fine (using InnoDB). When the row count gets noticeably larger, using Memcached (or any other in-memory key-value storage) gives marked performance improvements, but that is of course only possible if you can run the necessary daemon software on the server.

[Zeka]

@BitPoet Thanks for the info. I'm already on InnoDB.

I have Redis and Memcached on the server, but to be honest I have never used them as it's my first project where I hit some limits where probably it is a good thing to think about it. Could you recommend some reading about it or show some simple example of usage?

 

[BitPoet]

What I know has been pieced together from official docs and discovered through trial and error, so I can't provide a good link there. However, I've pushed a few pieces into a small module based on Memache that can be a (partial) stand in for WireCache as an example. You can find it on github as PwMemcache. It has just a few methods to set/get/delete cache entries and doesn't allow saving and restoring of full pages like WireCache does, but it does have a renderFile method similar to WireCache. Maybe that could be a starting point.

Most of the magic in using memory caches is

• using well thought out cache keys to avoid conflicting names (you're likely to use them more intensively than database caching)
• using reasonable expiry values (same as with database caches)
• preventing outdated data (e.g. by deleting entries when pages are changed, there's little 'automatic magic' there)
• taking care of special considerations when data is serialized and later unserialized, since behavior differs between solutions

How exactly that best goes depends on the features of the caching solution. Unlike WireCache, many in-memory solutions have no wildcard deletion operators, so you sometimes have to build workarounds. My module for example does this when you pass PwMemcache::expireSave to the renderFile method. The cache contains an extra array where all the entries are stored that have to be cleared when a page is saved. In a Pages::saved hook, this entry is read, all cache entries listed in the array are deleted and the entry is emptied.

[BitPoet]

I've dabbled a bit and adapted my cache module to work with Redis (working mostly on Windows, this was a bit of an adventure, but Windows Subsystem For Linux saved the day).

CacheRedis - a simple Redis cache interface for ProcessWire

There should be enough documentation to get started in the README.

In short, it has the following methods:

• $redis->fetch($key[, $expire, $func])
• $redis->store($key, $expire, $value)
• $redis->delete($key)
• $redis->flush()
• $redis->renderFile($file[, $expire[, $options]])