Jump to content

Cache Control - Clear all your caches in one place


MoritzLost
 Share

Recommended Posts

Process Cache Control

This module provides a simple solution to clearing all your cache layers at once, and an extensible interface to perform various cache-related actions.

The simple motivation behind this module was that I was tired of manually clearing caches in several places after deploying a change on a live site. The basic purpose of this module is a simple Clear all caches link in the Setup menu which clears out all caches, no matter where they hide. You can customize what exactly the module does through it's configuration menu:

  • Expire or delete all cache entries in the database, or selectively clear caches by namespace ($cache API)
  • Clear the the template render cache.
  • Clear out specific folders inside your site's cache directory (/site/assets/cache)
  • Clear the ProCache page render cache (if your site is using ProCache)
  • Refresh version strings for static assets to bust client-side browser caches (this requires some setup, see the full documentation for details).

This is the basic function of the module. However, you can also add different cache management action through the API and execute them through the module's interface. For this advanced usage, the module provides:

  • An interface to see all available cache actions and execute them.
  • A system log and logging output on the module page to see verify what the module is doing.
  • A CacheControlTools class with utility functions to clear out different caches.
  • An API to add cache actions, execute them programmatically and even modify the default action.
  • Permission management, allowing you granular control over which user roles can execute which actions.

The complete documentation can be found in the module's README.

Plans for improvements

If there is some interest in this, I plan to expand this to a more general cache management solution. I particular, I would like to add additional cache actions. Some ideas that came to mind:

  • Warming up the template render cache for publicly accessible pages.
  • Removing all active user sessions.

Let me know if you have more suggestions!

Links

Screenshots

1017168883_Screenshot2020-04-01at13_21_08.thumb.png.075305fb93e3b5aa1f4b41938c36cda0.png

Screenshot 2020-02-06 at 15.23.37.png

  • Like 16
Link to comment
Share on other sites

Hi @adrian, I didn't know that module existed, it's not in the directory, is it? Because I looked if something like this already existed and didn't find anything. Might've saved me some trouble 😅

That said, I had a quick look, here's what I found in comparing both modules:

  • ClearCacheAdmin exposes more options through the setup page, so there are available to everyone, whereas ProcessCacheControl has it's options in the module configuration, so they are only available to the developer.
  • ClearCacheAdmin has multiple links (and multiple options) on the process page, giving the user more fine control, whereas ProcessCacheControl goes for a simpler interface and bundles all those actions into one (configurable) action with a single button.
  • ProcessCacheControl can delete $cache entries without an expiry date, which ClearCacheAdmin doesn't as far as I can tell. ProcessCacheControl also lets you configure caches to be deleted by namespace, whereas ClearCacheAdmin offers each cache entry to be deleted individually (I think this has mostly to do with the module being a bit older, I believe $cache didn't have all those options back then).
  • The largest difference is my concept of "cache actions", which ClearCacheAdmin doesn't have. I'm not sure how useful that will actually be yet. I think if I can expand on the actions available by default, it will be pretty handy.
    • With ProcessCacheControl, you can add custom cache actions / profiles through the API, that may be useful depending on the use case.
    • Adding to that, ProcessCacheControl has permission checks on a per-action basis.
  • ProcessCacheControl can be modified and executed through the API. In particular, you can modify the default action and execute any action programmatically.
  • Like 5
  • Thanks 1
Link to comment
Share on other sites

As I was looking as well for a cache clearing solution through the admin and did not find one I just found out about PWGeeks from PW news letter. It seems very nice to have a global list of existing modules, however not everyone would still be aware from the community about the tool, so we might see similar modules being developed based on the customer/developers needs.

I will give a try to both cache clearing modules (if I may generalize the two) and see which one is suiting me more. At least some feedback can be shared.

Thanks @MoritzLost for the new module. It is the play time now 😉

  • Like 1
Link to comment
Share on other sites

Awesome! ClearCacheAdmin needed some love, this is definitely better IMO.

On 2/7/2020 at 3:12 AM, MoritzLost said:

In particular, can someone let me know if this module causes any problems with the ProCache module?

Will PM you on this, not sure how much I am allowed to talk about it here.

  • Like 1
  • Thanks 1
Link to comment
Share on other sites

Thanks everyone for the feedback! I just pushed a new version that resolves a bug that prevented installs on PHP versions <7.4.
PHP 7.1, 7.2 and 7.3 should work now!

Thanks @Mikie for your help! I have also found some documentation in the ProCache store page. I've added an option to clear the ProCache now as well, though I haven't included it in the new version because I can't test it out. Would you mind giving it a go? The ProCache integration is in the procache branch on Github (download link). If it's working as intended, it should add a new option to clear the ProCache in the module configuration, including a note saying whether ProCache is installed. If selected, the 'Clear all' action should use $procache->clearAll and log the result. It would also be good to know if non-superusers can use this, or if it's limited to the superuser. Thanks!

The module also appears to have been approved to the module directory, so it can now be downloaded directly through the backend using the module name ProcessCacheControl. Thanks @adrian (or is Ryan in charge of the module directory? 🙃)

  • Like 6
Link to comment
Share on other sites

Good thing I marked this as a beta release, because I'm still finding a lot of bugs 😅

I just pushed version 0.4.0. The module is now no longer autoloaded, because it's not really required anymore. This release splits the module into two seperate Modules: ProcessCacheControl and CacheControlTools. They are still installed alongside each other, and ProcessCacheControl will install CacheControlTools during install. For existing installs, you might have to reinstall the module, or install CacheControlTools manually after the upgrade. Also, the asset version management is now part of CacheControlTools, so the code has to be adjusted:

// version 0.3.0 and below
$CacheControlTools = $modules->get('ProcessCacheControl')->getTools();

// version 0.4.0
$CacheControlTools = $modules->get('CacheControlTools');

A bit of background for the split: I learned something new about the permission system. If a Process module defines a permission, the current user needs that permission not only to access the Process page in the admin, but also for the module to even be instantiated at all. Because of this, the asset management system didn't work for guest users. By splitting the repository into two modules, I can still have strict access control for the Process part of the module, while allowing the tools module to be instantiated even during requests from guest users.

Anyway, the module is now, let's say, nearing stability. I'd like to publish a stable release soon, so let me know if you find any more bugs!

  • Like 4
Link to comment
Share on other sites

  • 1 month later...

I just updated the module to version 0.5.0! I have now installed it on a couple of sites and it's pretty stable by now. But before I tag a stable 1.0.0 release I would really like to get the ProCache integration working.

ProCache integration - Testers wanted

If somebody has access to the ProCache module and would like to help me, I just need someone to test the ProCache integration!
To get started, download the procache branch of the module on Github and install it . There should be a new "ProCache" option in the module configuration. This field should also tell you whether you have ProCache installed. Check this option. Then go to the Process page through Setup -> Cache Control and click the "Clear all" button. The log output should clear the ProCache entirely. I'd like to know if you get any errors during any of those steps. Also, please check if the ProCache has been actually cleared.

Thanks!

Version 0.5.0

The new version provides some additional helper methods to check if users can access particular actions and improves the status messages displayed on the process page. In particular, there's a new static method to check if the user can access the module. This is important, because instantiating the module ($modules->get('ProcessCacheControl')) will throw an error if the user does not have the required permission. So this can now be checked with the static method ProcessCacheControl::canUseModule($user).

The complete Changelog can be found in the CHANGELOG.md in the repository.

  • Like 2
Link to comment
Share on other sites

5 hours ago, MoritzLost said:

ProCache integration - Testers wanted

I successfully tested it on one site several times (ca. 10x) with ProCaches between 50 and 700 pages! 🙂

Nice work!

  • Thanks 1
Link to comment
Share on other sites

12 hours ago, horst said:

I successfully tested it on one site several times (ca. 10x) with ProCaches between 50 and 700 pages! 🙂

Nice work!

@horst Awesome, thank you very much! I'll merge the branch into master soon then. I just need to update the documentation with all the new methods I added in the previous release, then I'll update the module to the first stable release.

I'm still thinking about other useful cache actions. I considered a "warm up" method that will just iterate over all the pages and perform anonymous HTTP requests to warm up the template render cache. Does something like this already exist? Would it be useful to anyone?
For smaller sites it would be trivial, but for larger sites the module might need some form of rate limiting, batch processing ... not sure if it's worth the effort?

  • Like 1
Link to comment
Share on other sites

59 minutes ago, MoritzLost said:

I considered a "warm up" method that will just iterate over all the pages and perform anonymous HTTP requests to warm up the template render cache. Does something like this already exist? Would it be useful to anyone?
For smaller sites it would be trivial, but for larger sites the module might need some form of rate limiting, batch processing ... not sure if it's worth the effort?

@MoritzLost A warmUp cache is definetly on the wishlist of some of us. This came up from time to time in forum posts. But with ProCache and larger sites it definetly is not trivial.

For smaller sites I have build some individual (hacky) solutions to drop and rebuild the ProCache pages, but for larger sites I manually use a local program that traverses a given website and collect information about broken links, redirects, etc.

I'm not sure how big the effort will / must be. I do know some sites (where I use my local tool) that have large archive content, that never will change. My PC time cache limit there is one year! It also could be infinite.

  • Like 2
Link to comment
Share on other sites

@horst Thanks for the feedback. Does warming up ProCache require any special consideration? I liked the approach of HTTP requests because of it's simplicity – i.e. it's virtually identical with a regular page visits, so the normal caching mechanisms can just do their thing. Or will that not work with ProCache for some reason?

Link to comment
Share on other sites

@MoritzLost I think it is "the best way!" to use (guest) HTTP requests to warm up a ProCache.  Thats why I really like the use of this crawler tool. It starts at the homepage, determine links to the homepages childpages, fetch and parse them, and so on.

I think the real problem is to get all "real public URLs" from within PW, as I have sites that use many many pages that have a public state, but in real they are only single sections of a page. So, maybe thats my private problem, but I'm not sure if there is a way to detect which pages / parent pages are official / public URLs. I think if a warmup function can get passed a optional URL list or a PageArray, that overwrites a default behavior, this would be a great benefit for all out there, that uses MVC and other individual approaches. 

  • Like 1
Link to comment
Share on other sites

@horst Hm, I hadn't considered that, that makes it more difficult indeed. Come to think of it, most of my sites would have this issue as well - for example, I use a non-standard pagination approach with URL segments, those wouldn't be caught by just using the public URL of the base page; only the first page would be reached. I guess webcrawling would be a way around that, though that is a real can of worms ...

Maybe a good compromise would be making the process hookable, like you said. Maybe the method can call a hookable method that returns an array of URLs for each page; by default it returns only the normal URLs, and additional URLs with additional parameters, url segments et c. can be added through hooks. I'll give this a go when I have the time!

  • Like 1
Link to comment
Share on other sites

8 hours ago, MoritzLost said:

I considered a "warm up" method that will just iterate over all the pages and perform anonymous HTTP requests to warm up the template render cache. Does something like this already exist? Would it be useful to anyone?

I think all caching "solutions" would somehow benefit with "warm-up" (you can try a new version of AIOM that mimics how ProCache works). Most crawlers do what you want to do more or less successfully (with some gotchas you and Horst mentioned), but it would certainly look more "professional" and it would be integrated into PW.

  • Like 2
Link to comment
Share on other sites

  • 2 weeks later...
2 hours ago, David Karich said:

@MoritzLost Nice work! Tested: the page cache is deleted, but not the compiled assets (js, css) of ProCache. PW DEV 3.0.153, ProCache latest, Win and Linux, PHP 7.3, PHP 7.4

Thanks @David Karich! Hm, right now the module is just calling $procache->clearAll() which is supposed to clear everything according to the ProCache store page. Is there another method to clear the minified assets as well? I checked all documentation I could find, a couple of ideas:

  1. I don't know how ProCache works with minified assets, but according to this blog post any changes to the CSS & JS files should be picked up automatically, so maybe it's not necessary to clear those? Though of course that is not ideal for a cache clear button 🙂
  2. Judging by the last screenshot on this blog post, ProCache stores minified assets inside /site/assets/pwpc, or is that out of date? Currently, ProcessCacheControl only supports clearing out folders inside /site/assets/cache (so it can't accidentally nuke the files directory because of a misconfigured setting ...). Maybe I could extend that to also allow clearing /site/assets/pwpc.  Is that directory path static or can it be changed?

Anyway, I'll put out a stable release with updated documentation & the ProCache integration soon! Hopefully the minification issue can also be resolved then.

  • Like 1
Link to comment
Share on other sites

I have now integrated the ProCache branch into the module and released it as version 1.0.0!

  • I have tested the module on multiple sites and it's pretty stable now. But if you do run into bugs, let me know!
  • The ProCache integration is now included, so you can clear out the ProCache page render cache alongside all other caches. Hopefully we can resolve the issue with minified assets, see the previous post.
  • The documentation is now updated with information on all the new methods added in the previous release.

Let me know if something isn't working for you!

  • Like 4
Link to comment
Share on other sites

  • 1 month later...
  • 9 months later...

ProcessCacheControl version 1.1.0 released

I just released an updated version of this module which fixes some issues with the asset version management feature. In particular:

  • When using CacheControlTools::getAssetVersion while no asset version existed (for the specified category), the implicitly generated version incorrectly expired after one day. This is now fixed, so asset versions will not expirate anymore until manually cleared.
  • Asset versions are no stored using WireCache::expireNever instead of WireCache::expireReserved. This means they will be deleted when using $cache->deleteAll(), when previously you had to explicity specify them by namespace to clear the cached versions.

Links:

  • Like 2
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By Robin S
      This module lets you add some custom menu items to the main admin menu, and you can set the dropdown links dynamically in a hook if needed.
      Sidenote: the module config uses some repeatable/sortable rows for the child link settings, similar to the ProFields Table interface. The data gets saved as JSON in a hidden textarea field. Might be interesting to other module developers?
      Custom Admin Menus
      Adds up to three custom menu items with optional dropdowns to the main admin menu.
      The menu items can link to admin pages, front-end pages, or pages on external websites.
      The links can be set to open in a new browser tab, and child links in the dropdown can be given an icon.
      Requires ProcessWire v3.0.178 or newer.
      Screenshots
      Example of menu items

      Module config for the menus

      Link list shown when parent menu item is not given a URL

      Advanced
      Setting child menu items dynamically
      If needed you can set the child menu items dynamically using a hook.
      Example:
      $wire->addHookAfter('CustomAdminMenus::getMenuChildren', function(HookEvent $event) { // The menu number is the first argument $menu_number = $event->arguments(0); if($menu_number === 1) { $colours = $event->wire()->pages->findRaw('template=colour', ['title', 'url', 'page_icon']); $children = []; foreach($colours as $colour) { // Each child item should be an array with the following keys $children[] = [ 'icon' => $colour['page_icon'], 'label' => $colour['title'], 'url' => $colour['url'], 'newtab' => false, ]; } $event->return = $children; } }); Create multiple levels of flyout menus
      It's also possible to create multiple levels of flyout submenus using a hook.

      For each level a submenu can be defined in a "children" item. Example:
      $wire->addHookAfter('CustomAdminMenus::getMenuChildren', function(HookEvent $event) { // The menu number is the first argument $menu_number = $event->arguments(0); if($menu_number === 1) { $children = [ [ 'icon' => 'adjust', 'label' => 'One', 'url' => '/one/', 'newtab' => false, ], [ 'icon' => 'anchor', 'label' => 'Two', 'url' => '/two/', 'newtab' => false, 'children' => [ [ 'icon' => 'child', 'label' => 'Red', 'url' => '/red/', 'newtab' => false, ], [ 'icon' => 'bullhorn', 'label' => 'Green', 'url' => '/green/', 'newtab' => false, 'children' => [ [ 'icon' => 'wifi', 'label' => 'Small', 'url' => '/small/', 'newtab' => true, ], [ 'icon' => 'codepen', 'label' => 'Medium', 'url' => '/medium/', 'newtab' => false, ], [ 'icon' => 'cogs', 'label' => 'Large', 'url' => '/large/', 'newtab' => false, ], ] ], [ 'icon' => 'futbol-o', 'label' => 'Blue', 'url' => '/blue/', 'newtab' => true, ], ] ], [ 'icon' => 'hand-o-left', 'label' => 'Three', 'url' => '/three/', 'newtab' => false, ], ]; $event->return = $children; } }); Showing/hiding menus according to user role
      You can determine which menu items can be seen by a role by checking the user's role in the hook.
      For example, if a user has or lacks a role you could include different child menu items in the hook return value. Or if you want to conditionally hide a custom menu altogether you can set the return value to false. Example:
      $wire->addHookAfter('CustomAdminMenus::getMenuChildren', function(HookEvent $event) { // The menu number is the first argument $menu_number = $event->arguments(0); $user = $event->wire()->user; // For custom menu number 1... if($menu_number === 1) { // ...if user does not have some particular role... if(!$user->hasRole('foo')) { // ...do not show the menu $event->return = false; } } });  
      https://github.com/Toutouwai/CustomAdminMenus
      https://processwire.com/modules/custom-admin-menus/
    • By tcnet
      This module for ProcessWire sends a notification email for each failed login attempt. Similar modules exists already in the module directory of ProcessWire. However, this module is designed to notify, even if specified user doesn't exist.
      Settings
      The settings for this module are located in the menu Modules=>Configure=>LoginFailNotifier.
      Notification email
      Specifies the email address to which the notification emails should be sent.
        Email subject
      Specifies the subject line for the notification email.
        Post variables
      Specifies the $_POST variables to be included in the notification email. Each variable must be separated by a comma. For example: login_name,login_pass
        Server variables
      Specifies the $_SERVER variables to be included in the notification email. Each variable must be separated by a comma. For example: REMOTE_ADDR,HTTP_USER_AGENT
      Link to ProcessWire module directory:
      https://processwire.com/modules/login-fail-notifier/
      Link to github.com:
      https://github.com/techcnet/LoginFailNotifier
    • By Fokke
      ProcessWire 3.x markup module for rendering meta tags in HTML document head section. Note that this module is not a full-blown SEO solution, but rather a simple tool for rendering meta tags based on module configuration. Adding custom meta tags is also supported.
      Built-in meta tags
      The following meta tags are supported out-of-the-box:
      Document title consisting of page title and site name Character set Canonical Viewport Description Keywords Hreflang tags Open Graph og:title og:site_name og:type og:url og:description og:image og:image:width og:image:height Twitter meta tags twitter:card twitter:site twitter:creator twitter:title twitter:description twitter:image Facebook meta tags fb:app_id The full documentation with configurable options can be found here: https://github.com/Fokke-/MarkupMetadata
       
      Requirements:
      ProcessWire>=3.0.0 PHP >=7.1 Installation using Composer
      composer require fokke/markup-metadata Manual installation
      Download latest version from https://github.com/Fokke-/MarkupMetadata/archive/master.zip Extract module files to site/modules/MarkupMetadata directory.
    • By m.sieber
      ITRK-Service for ProcessWire
      Module for the automated transfer of imprint, data protection declaration and terms and conditions from IT-Recht Kanzlei to your ProcessWire installation
      What is ITRK Service for ProcessWire?
      ITRK-Service for ProcessWire is a free module for ProcessWire CMS. It provides an interface to the update service of IT-Recht Kanzlei, via which the legal texts of your online presence are automatically updated. In this way, the texts remain legally secure and warning-proof in the long term. Imprint, data protection declaration, revocation and general terms and conditions are currently supported.
      You can find our documentation (in german language) here: https://www.pupit.de/itrk-service-for-processwire/dokumentation/

      Download: https://www.pupit.de/itrk-service-for-processwire/
      Github: https://github.com/pupit-de/pwItrkServiceConnector
    • By LuisM
      Symprowire is a PHP MVC Framework based and built on Symfony using ProcessWire 3.x as DBAL and Service-Provider
      It acts as a Drop-In Replacement Module to handle the Request/Response outside the ProcessWire Admin. Even tough Symfony or any other mature MVC Framework could be intimidating at first, Symprowire tries to abstract Configuration and Symfony Internals away as much as possible to give you a quick start and lift the heavy work for you.
      The main Goal is to give an easy path to follow an MVC Approach during development with ProcessWire and open up the available eco-system.
      You can find the GitHub Repo and more Information here: https://github.com/Luis85/symprowire
      Documentation
      The Symprowire Wiki https://github.com/Luis85/symprowire/wiki How to create a simple Blog with Symprowire https://github.com/Luis85/symprowire/wiki/Symprowire-Blog-Tutorial Last Update
      16.07.2021 // RC 1 v0.6.0 centralized ProcessWire access trough out the Application by wrapping to a Service https://github.com/Luis85/symprowire/releases/tag/v0.6.0-rc-1 Requirements
      PHP ^7.4 Fresh ProcessWire ^3.0.181 with a Blank Profile Composer 2 (v1 should work, not recommended) The usual Symfony Requirements Features
      Twig Dependency Injection Monolog for Symprowire Support for .env YAML Configuration Symfony Console and Console Commands Symfony Webprofiler Full ProcessWire access inside your Controller and Services Webpack Encore support Caveats
      Symfony is no small Framework and will come with a price in terms of Memory Usage and added Overhead. To give you a taste I installed Tracy Debugger alongside to compare ProcessWire profiling with the included Symfony Webprofiler

      So in a fresh install Symprowire would atleast add another 2MB of Memory usage and around 40ms in response time, should be less in production due to the added overhead of the Webprofiler in dev env
       
×
×
  • Create New...