Jump to content

Cache Control - Clear all your caches in one place


MoritzLost
 Share

Recommended Posts

Process Cache Control

This module provides a simple solution to clearing all your cache layers at once, and an extensible interface to perform various cache-related actions.

The simple motivation behind this module was that I was tired of manually clearing caches in several places after deploying a change on a live site. The basic purpose of this module is a simple Clear all caches link in the Setup menu which clears out all caches, no matter where they hide. You can customize what exactly the module does through it's configuration menu:

  • Expire or delete all cache entries in the database, or selectively clear caches by namespace ($cache API)
  • Clear the the template render cache.
  • Clear out specific folders inside your site's cache directory (/site/assets/cache)
  • Clear the ProCache page render cache (if your site is using ProCache)
  • Refresh version strings for static assets to bust client-side browser caches (this requires some setup, see the full documentation for details).

This is the basic function of the module. However, you can also add different cache management action through the API and execute them through the module's interface. For this advanced usage, the module provides:

  • An interface to see all available cache actions and execute them.
  • A system log and logging output on the module page to see verify what the module is doing.
  • A CacheControlTools class with utility functions to clear out different caches.
  • An API to add cache actions, execute them programmatically and even modify the default action.
  • Permission management, allowing you granular control over which user roles can execute which actions.

The complete documentation can be found in the module's README.

Plans for improvements

If there is some interest in this, I plan to expand this to a more general cache management solution. I particular, I would like to add additional cache actions. Some ideas that came to mind:

  • Warming up the template render cache for publicly accessible pages.
  • Removing all active user sessions.

Let me know if you have more suggestions!

Links

Screenshots

1017168883_Screenshot2020-04-01at13_21_08.thumb.png.075305fb93e3b5aa1f4b41938c36cda0.png

Screenshot 2020-02-06 at 15.23.37.png

  • Like 16
Link to comment
Share on other sites

Hi @adrian, I didn't know that module existed, it's not in the directory, is it? Because I looked if something like this already existed and didn't find anything. Might've saved me some trouble 😅

That said, I had a quick look, here's what I found in comparing both modules:

  • ClearCacheAdmin exposes more options through the setup page, so there are available to everyone, whereas ProcessCacheControl has it's options in the module configuration, so they are only available to the developer.
  • ClearCacheAdmin has multiple links (and multiple options) on the process page, giving the user more fine control, whereas ProcessCacheControl goes for a simpler interface and bundles all those actions into one (configurable) action with a single button.
  • ProcessCacheControl can delete $cache entries without an expiry date, which ClearCacheAdmin doesn't as far as I can tell. ProcessCacheControl also lets you configure caches to be deleted by namespace, whereas ClearCacheAdmin offers each cache entry to be deleted individually (I think this has mostly to do with the module being a bit older, I believe $cache didn't have all those options back then).
  • The largest difference is my concept of "cache actions", which ClearCacheAdmin doesn't have. I'm not sure how useful that will actually be yet. I think if I can expand on the actions available by default, it will be pretty handy.
    • With ProcessCacheControl, you can add custom cache actions / profiles through the API, that may be useful depending on the use case.
    • Adding to that, ProcessCacheControl has permission checks on a per-action basis.
  • ProcessCacheControl can be modified and executed through the API. In particular, you can modify the default action and execute any action programmatically.
  • Like 5
  • Thanks 1
Link to comment
Share on other sites

As I was looking as well for a cache clearing solution through the admin and did not find one I just found out about PWGeeks from PW news letter. It seems very nice to have a global list of existing modules, however not everyone would still be aware from the community about the tool, so we might see similar modules being developed based on the customer/developers needs.

I will give a try to both cache clearing modules (if I may generalize the two) and see which one is suiting me more. At least some feedback can be shared.

Thanks @MoritzLost for the new module. It is the play time now 😉

  • Like 1
Link to comment
Share on other sites

Awesome! ClearCacheAdmin needed some love, this is definitely better IMO.

On 2/7/2020 at 3:12 AM, MoritzLost said:

In particular, can someone let me know if this module causes any problems with the ProCache module?

Will PM you on this, not sure how much I am allowed to talk about it here.

  • Like 1
  • Thanks 1
Link to comment
Share on other sites

Thanks everyone for the feedback! I just pushed a new version that resolves a bug that prevented installs on PHP versions <7.4.
PHP 7.1, 7.2 and 7.3 should work now!

Thanks @Mikie for your help! I have also found some documentation in the ProCache store page. I've added an option to clear the ProCache now as well, though I haven't included it in the new version because I can't test it out. Would you mind giving it a go? The ProCache integration is in the procache branch on Github (download link). If it's working as intended, it should add a new option to clear the ProCache in the module configuration, including a note saying whether ProCache is installed. If selected, the 'Clear all' action should use $procache->clearAll and log the result. It would also be good to know if non-superusers can use this, or if it's limited to the superuser. Thanks!

The module also appears to have been approved to the module directory, so it can now be downloaded directly through the backend using the module name ProcessCacheControl. Thanks @adrian (or is Ryan in charge of the module directory? 🙃)

  • Like 6
Link to comment
Share on other sites

Good thing I marked this as a beta release, because I'm still finding a lot of bugs 😅

I just pushed version 0.4.0. The module is now no longer autoloaded, because it's not really required anymore. This release splits the module into two seperate Modules: ProcessCacheControl and CacheControlTools. They are still installed alongside each other, and ProcessCacheControl will install CacheControlTools during install. For existing installs, you might have to reinstall the module, or install CacheControlTools manually after the upgrade. Also, the asset version management is now part of CacheControlTools, so the code has to be adjusted:

// version 0.3.0 and below
$CacheControlTools = $modules->get('ProcessCacheControl')->getTools();

// version 0.4.0
$CacheControlTools = $modules->get('CacheControlTools');

A bit of background for the split: I learned something new about the permission system. If a Process module defines a permission, the current user needs that permission not only to access the Process page in the admin, but also for the module to even be instantiated at all. Because of this, the asset management system didn't work for guest users. By splitting the repository into two modules, I can still have strict access control for the Process part of the module, while allowing the tools module to be instantiated even during requests from guest users.

Anyway, the module is now, let's say, nearing stability. I'd like to publish a stable release soon, so let me know if you find any more bugs!

  • Like 4
Link to comment
Share on other sites

  • 1 month later...

I just updated the module to version 0.5.0! I have now installed it on a couple of sites and it's pretty stable by now. But before I tag a stable 1.0.0 release I would really like to get the ProCache integration working.

ProCache integration - Testers wanted

If somebody has access to the ProCache module and would like to help me, I just need someone to test the ProCache integration!
To get started, download the procache branch of the module on Github and install it . There should be a new "ProCache" option in the module configuration. This field should also tell you whether you have ProCache installed. Check this option. Then go to the Process page through Setup -> Cache Control and click the "Clear all" button. The log output should clear the ProCache entirely. I'd like to know if you get any errors during any of those steps. Also, please check if the ProCache has been actually cleared.

Thanks!

Version 0.5.0

The new version provides some additional helper methods to check if users can access particular actions and improves the status messages displayed on the process page. In particular, there's a new static method to check if the user can access the module. This is important, because instantiating the module ($modules->get('ProcessCacheControl')) will throw an error if the user does not have the required permission. So this can now be checked with the static method ProcessCacheControl::canUseModule($user).

The complete Changelog can be found in the CHANGELOG.md in the repository.

  • Like 2
Link to comment
Share on other sites

5 hours ago, MoritzLost said:

ProCache integration - Testers wanted

I successfully tested it on one site several times (ca. 10x) with ProCaches between 50 and 700 pages! 🙂

Nice work!

  • Thanks 1
Link to comment
Share on other sites

12 hours ago, horst said:

I successfully tested it on one site several times (ca. 10x) with ProCaches between 50 and 700 pages! 🙂

Nice work!

@horst Awesome, thank you very much! I'll merge the branch into master soon then. I just need to update the documentation with all the new methods I added in the previous release, then I'll update the module to the first stable release.

I'm still thinking about other useful cache actions. I considered a "warm up" method that will just iterate over all the pages and perform anonymous HTTP requests to warm up the template render cache. Does something like this already exist? Would it be useful to anyone?
For smaller sites it would be trivial, but for larger sites the module might need some form of rate limiting, batch processing ... not sure if it's worth the effort?

  • Like 1
Link to comment
Share on other sites

59 minutes ago, MoritzLost said:

I considered a "warm up" method that will just iterate over all the pages and perform anonymous HTTP requests to warm up the template render cache. Does something like this already exist? Would it be useful to anyone?
For smaller sites it would be trivial, but for larger sites the module might need some form of rate limiting, batch processing ... not sure if it's worth the effort?

@MoritzLost A warmUp cache is definetly on the wishlist of some of us. This came up from time to time in forum posts. But with ProCache and larger sites it definetly is not trivial.

For smaller sites I have build some individual (hacky) solutions to drop and rebuild the ProCache pages, but for larger sites I manually use a local program that traverses a given website and collect information about broken links, redirects, etc.

I'm not sure how big the effort will / must be. I do know some sites (where I use my local tool) that have large archive content, that never will change. My PC time cache limit there is one year! It also could be infinite.

  • Like 2
Link to comment
Share on other sites

@horst Thanks for the feedback. Does warming up ProCache require any special consideration? I liked the approach of HTTP requests because of it's simplicity – i.e. it's virtually identical with a regular page visits, so the normal caching mechanisms can just do their thing. Or will that not work with ProCache for some reason?

Link to comment
Share on other sites

@MoritzLost I think it is "the best way!" to use (guest) HTTP requests to warm up a ProCache.  Thats why I really like the use of this crawler tool. It starts at the homepage, determine links to the homepages childpages, fetch and parse them, and so on.

I think the real problem is to get all "real public URLs" from within PW, as I have sites that use many many pages that have a public state, but in real they are only single sections of a page. So, maybe thats my private problem, but I'm not sure if there is a way to detect which pages / parent pages are official / public URLs. I think if a warmup function can get passed a optional URL list or a PageArray, that overwrites a default behavior, this would be a great benefit for all out there, that uses MVC and other individual approaches. 

  • Like 1
Link to comment
Share on other sites

@horst Hm, I hadn't considered that, that makes it more difficult indeed. Come to think of it, most of my sites would have this issue as well - for example, I use a non-standard pagination approach with URL segments, those wouldn't be caught by just using the public URL of the base page; only the first page would be reached. I guess webcrawling would be a way around that, though that is a real can of worms ...

Maybe a good compromise would be making the process hookable, like you said. Maybe the method can call a hookable method that returns an array of URLs for each page; by default it returns only the normal URLs, and additional URLs with additional parameters, url segments et c. can be added through hooks. I'll give this a go when I have the time!

  • Like 1
Link to comment
Share on other sites

8 hours ago, MoritzLost said:

I considered a "warm up" method that will just iterate over all the pages and perform anonymous HTTP requests to warm up the template render cache. Does something like this already exist? Would it be useful to anyone?

I think all caching "solutions" would somehow benefit with "warm-up" (you can try a new version of AIOM that mimics how ProCache works). Most crawlers do what you want to do more or less successfully (with some gotchas you and Horst mentioned), but it would certainly look more "professional" and it would be integrated into PW.

  • Like 2
Link to comment
Share on other sites

  • 2 weeks later...
2 hours ago, David Karich said:

@MoritzLost Nice work! Tested: the page cache is deleted, but not the compiled assets (js, css) of ProCache. PW DEV 3.0.153, ProCache latest, Win and Linux, PHP 7.3, PHP 7.4

Thanks @David Karich! Hm, right now the module is just calling $procache->clearAll() which is supposed to clear everything according to the ProCache store page. Is there another method to clear the minified assets as well? I checked all documentation I could find, a couple of ideas:

  1. I don't know how ProCache works with minified assets, but according to this blog post any changes to the CSS & JS files should be picked up automatically, so maybe it's not necessary to clear those? Though of course that is not ideal for a cache clear button 🙂
  2. Judging by the last screenshot on this blog post, ProCache stores minified assets inside /site/assets/pwpc, or is that out of date? Currently, ProcessCacheControl only supports clearing out folders inside /site/assets/cache (so it can't accidentally nuke the files directory because of a misconfigured setting ...). Maybe I could extend that to also allow clearing /site/assets/pwpc.  Is that directory path static or can it be changed?

Anyway, I'll put out a stable release with updated documentation & the ProCache integration soon! Hopefully the minification issue can also be resolved then.

  • Like 1
Link to comment
Share on other sites

I have now integrated the ProCache branch into the module and released it as version 1.0.0!

  • I have tested the module on multiple sites and it's pretty stable now. But if you do run into bugs, let me know!
  • The ProCache integration is now included, so you can clear out the ProCache page render cache alongside all other caches. Hopefully we can resolve the issue with minified assets, see the previous post.
  • The documentation is now updated with information on all the new methods added in the previous release.

Let me know if something isn't working for you!

  • Like 4
Link to comment
Share on other sites

  • 1 month later...
  • 9 months later...

ProcessCacheControl version 1.1.0 released

I just released an updated version of this module which fixes some issues with the asset version management feature. In particular:

  • When using CacheControlTools::getAssetVersion while no asset version existed (for the specified category), the implicitly generated version incorrectly expired after one day. This is now fixed, so asset versions will not expirate anymore until manually cleared.
  • Asset versions are no stored using WireCache::expireNever instead of WireCache::expireReserved. This means they will be deleted when using $cache->deleteAll(), when previously you had to explicity specify them by namespace to clear the cached versions.

Links:

  • Like 2
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Similar Content

    • By MarkE
      This fieldtype and inputfield bundle was built for storing measurement values within a field, rendering them in a variety of formats and converting them to other units or otherwise modifying them via the API.
      The API consists of a number of predefined functions, some of which include...
      render() for rendering the measurement object, valueAs() for converting the value to another unit value, convertTo() for converting the whole measurement object to different units, and add() and subtract() for creating a new measurement object from the sum or difference between two other objects. In the admin the inputfield includes a checkbox (which can be optionally disabled) for converting values on page save. For an example if a value was typed in as centimeters, the unit was changed to metres, and the page saved with this checkbox selected, said value would be automatically converted so that e.g. 170 cm becomes 1.7 m.
      A simple length field using Fieldtype Measurement and Inputfield Measurement.
      Combination units (e.g. feet and inches) are also supported.
      Please note that this module is 'proof of concept' at the moment - there are limited units available and quite a lot of code tidying to do. More units will be added shortly.
      See the GitHub at https://github.com/MetaTunes/FieldtypeMeasurement for full details and updates.
    • By tcnet
      File Manager for ProcessWire is a module to manager files and folders from the CMS backend. It supports creating, deleting, renaming, packing, unpacking, uploading, downloading and editing of files and folders. The integrated code editor ACE supports highlighting of all common programming languages.
      https://github.com/techcnet/ProcessFileManager

      Warning
      This module is probably the most powerful module. You might destroy your processwire installation if you don't exactly know what you doing. Be careful and use it at your own risk!
      ACE code editor
      This module uses ACE code editor available from: https://github.com/ajaxorg/ace

      Dragscroll
      This module uses the JavaScript dragscroll available from: http://github.com/asvd/dragscroll. Dragscroll adds the ability to drag the table horizontally with the mouse pointer.
      PHP File Manager
      This module uses a modified version of PHP File Manager available from: https://github.com/alexantr/filemanager
       
    • By tcnet
      This module implements the website live chat service from tawk.to. Actually the module doesn't have to do much. It just need to inserted a few lines of JavaScript just before the closing body tag </body> on each side. However, the module offers additional options to display the widget only on certain pages.
      Create an account
      Visit https://www.tawk.to and create an account. It's free! At some point you will reach a page where you can copy the required JavaScript-code.

      Open the module settings and paste the JavaScript-code into the field as shown below. Click "Submit" and that's all.

      Open the module settings
      The settings for this module are located int the menu Modules=>Configure=>LiveChatTawkTo.

       
    • By tcnet
      Session Viewer is a module for ProcessWire to list session files and display session data. This module is helpful to display the session data of a specific session or to kick out a logged in user by simply delete his session file. After installation the module is available in the Setup menu.

      The following conditions must be met for the module to work properly:
      Session files
      Session data must be stored in session files, which is the default way in ProcessWire. Sessions stored in the database are not supported by this module. The path to the directory where the session files are stored must be declared in the ProcessWire configuration which is by default: site/assets/sessions.
      Serialize handler
      In order to transform session data easier back to a PHP array, the session data is stored serialized. PHP offers a way to declare a custom serialize handler. This module supports only the default serialize handlers: php, php_binary and php_serialize. WDDX was dropped in PHP 7.4.0 and is therefore not supported by this module as well as any other custom serialize handler. Which serialize handler is actually used you can find out in the module configuration which is available under Modules=>Configure=>SessionViewer.

      Session data
      The session data can be displayed in two different ways. PHP's default output for arrays print_r() or by default for this module nice_r() offered on github: https://github.com/uuf6429/nice_r. There is a setting in the module configuration if someone prefers print_r(). Apart from the better handling and overview of the folded session data the output of nice_r() looks indeed nicer.

      Links
      ProcessWire module directory
      github.com
    • By Robin S
      Repeater Easy Sort
      Adds a compact "easy-sort" mode to Repeater and Repeater Matrix, making those fields easier to sort when there are a large number of items.
      The module also enhances Repeater Matrix by allowing a colour to be set for each matrix type. This colour is used in the item headers and in the "add new" links, to help visually distinguish different matrix types in the inputfield.
      Screencasts
      A Repeater field

      A Repeater Matrix field with custom header colours

      Easy-sort mode
      Each Repeater/Matrix item gets an double-arrow icon in the item header. Click this icon to enter easy-sort mode.
      While in easy-sort mode:
      The items will reduce in width so that more items can be shown on the screen at once. The minimum width is configurable in the field settings. Any items that were in an open state are collapsed, but when you exit easy-sort mode the previously open items will be reopened. You can drag an item left/right/up/down to sort it within the items. The item that you clicked the icon for is shown with a black background. This makes it easier to find the item you want to move in easy-sort mode. You can click an item header to open the item. An "Exit easy-sort mode" button appears at the bottom of the inputfield. Configuration
      In the field settings for Repeater and Repeater Matrix fields you can define a minimum width in pixels for items in easy-sort mode. While in easy-sort mode the items will be sized to neatly fill the available width on any screen size but will never be narrower than the width you set here.
      In the field settings for Repeater Matrix you can define a custom header colour for each matrix type using an HTML "color" type input. The default colour for this type of input is black, so when black is selected in the input it means that no custom colour will be applied to the header.
      Exclusions
      The easy-sort mode is only possible on Repeater/Matrix fields that do not use the "item depth" option.
       
      https://github.com/Toutouwai/RepeaterEasySort
      https://processwire.com/modules/repeater-easy-sort/
×
×
  • Create New...