Jump to content
MoritzLost

Cache Control - Clear all your caches in one place

Recommended Posts

Process Cache Control

This module provides a simple solution to clearing all your cache layers at once, and an extensible interface to perform various cache-related actions.

The simple motivation behind this module was that I was tired of manually clearing caches in several places after deploying a change on a live site. The basic purpose of this module is a simple Clear all caches link in the Setup menu which clears out all caches, no matter where they hide. You can customize what exactly the module does through it's configuration menu:

  • Expire or delete all cache entries in the database, or selectively clear caches by namespace ($cache API)
  • Clear the the template render cache.
  • Clear out specific folders inside your site's cache directory (/site/assets/cache)
  • Clear the ProCache page render cache (if your site is using ProCache)
  • Refresh version strings for static assets to bust client-side browser caches (this requires some setup, see the full documentation for details).

This is the basic function of the module. However, you can also add different cache management action through the API and execute them through the module's interface. For this advanced usage, the module provides:

  • An interface to see all available cache actions and execute them.
  • A system log and logging output on the module page to see verify what the module is doing.
  • A CacheControlTools class with utility functions to clear out different caches.
  • An API to add cache actions, execute them programmatically and even modify the default action.
  • Permission management, allowing you granular control over which user roles can execute which actions.

The complete documentation can be found in the module's README.

Plans for improvements

If there is some interest in this, I plan to expand this to a more general cache management solution. I particular, I would like to add additional cache actions. Some ideas that came to mind:

  • Warming up the template render cache for publicly accessible pages.
  • Removing all active user sessions.

Let me know if you have more suggestions!

Links

Screenshots

1017168883_Screenshot2020-04-01at13_21_08.thumb.png.075305fb93e3b5aa1f4b41938c36cda0.png

Screenshot 2020-02-06 at 15.23.37.png

  • Like 14

Share this post


Link to post
Share on other sites

Hi @adrian, I didn't know that module existed, it's not in the directory, is it? Because I looked if something like this already existed and didn't find anything. Might've saved me some trouble 😅

That said, I had a quick look, here's what I found in comparing both modules:

  • ClearCacheAdmin exposes more options through the setup page, so there are available to everyone, whereas ProcessCacheControl has it's options in the module configuration, so they are only available to the developer.
  • ClearCacheAdmin has multiple links (and multiple options) on the process page, giving the user more fine control, whereas ProcessCacheControl goes for a simpler interface and bundles all those actions into one (configurable) action with a single button.
  • ProcessCacheControl can delete $cache entries without an expiry date, which ClearCacheAdmin doesn't as far as I can tell. ProcessCacheControl also lets you configure caches to be deleted by namespace, whereas ClearCacheAdmin offers each cache entry to be deleted individually (I think this has mostly to do with the module being a bit older, I believe $cache didn't have all those options back then).
  • The largest difference is my concept of "cache actions", which ClearCacheAdmin doesn't have. I'm not sure how useful that will actually be yet. I think if I can expand on the actions available by default, it will be pretty handy.
    • With ProcessCacheControl, you can add custom cache actions / profiles through the API, that may be useful depending on the use case.
    • Adding to that, ProcessCacheControl has permission checks on a per-action basis.
  • ProcessCacheControl can be modified and executed through the API. In particular, you can modify the default action and execute any action programmatically.
  • Like 5
  • Thanks 1

Share this post


Link to post
Share on other sites

No, it's not in the directory. You should really take a look at @netcarver's awesome: https://pwgeeks.com/ - just type "cache" in there and you'll find some other cache related tools.

Thanks for the rundown of differences - that will be really helpful.

  • Like 5

Share this post


Link to post
Share on other sites

As I was looking as well for a cache clearing solution through the admin and did not find one I just found out about PWGeeks from PW news letter. It seems very nice to have a global list of existing modules, however not everyone would still be aware from the community about the tool, so we might see similar modules being developed based on the customer/developers needs.

I will give a try to both cache clearing modules (if I may generalize the two) and see which one is suiting me more. At least some feedback can be shared.

Thanks @MoritzLost for the new module. It is the play time now 😉

  • Like 1

Share this post


Link to post
Share on other sites

Awesome! ClearCacheAdmin needed some love, this is definitely better IMO.

On 2/7/2020 at 3:12 AM, MoritzLost said:

In particular, can someone let me know if this module causes any problems with the ProCache module?

Will PM you on this, not sure how much I am allowed to talk about it here.

  • Like 1
  • Thanks 1

Share this post


Link to post
Share on other sites

Thanks everyone for the feedback! I just pushed a new version that resolves a bug that prevented installs on PHP versions <7.4.
PHP 7.1, 7.2 and 7.3 should work now!

Thanks @Mikie for your help! I have also found some documentation in the ProCache store page. I've added an option to clear the ProCache now as well, though I haven't included it in the new version because I can't test it out. Would you mind giving it a go? The ProCache integration is in the procache branch on Github (download link). If it's working as intended, it should add a new option to clear the ProCache in the module configuration, including a note saying whether ProCache is installed. If selected, the 'Clear all' action should use $procache->clearAll and log the result. It would also be good to know if non-superusers can use this, or if it's limited to the superuser. Thanks!

The module also appears to have been approved to the module directory, so it can now be downloaded directly through the backend using the module name ProcessCacheControl. Thanks @adrian (or is Ryan in charge of the module directory? 🙃)

  • Like 6

Share this post


Link to post
Share on other sites

Good thing I marked this as a beta release, because I'm still finding a lot of bugs 😅

I just pushed version 0.4.0. The module is now no longer autoloaded, because it's not really required anymore. This release splits the module into two seperate Modules: ProcessCacheControl and CacheControlTools. They are still installed alongside each other, and ProcessCacheControl will install CacheControlTools during install. For existing installs, you might have to reinstall the module, or install CacheControlTools manually after the upgrade. Also, the asset version management is now part of CacheControlTools, so the code has to be adjusted:

// version 0.3.0 and below
$CacheControlTools = $modules->get('ProcessCacheControl')->getTools();

// version 0.4.0
$CacheControlTools = $modules->get('CacheControlTools');

A bit of background for the split: I learned something new about the permission system. If a Process module defines a permission, the current user needs that permission not only to access the Process page in the admin, but also for the module to even be instantiated at all. Because of this, the asset management system didn't work for guest users. By splitting the repository into two modules, I can still have strict access control for the Process part of the module, while allowing the tools module to be instantiated even during requests from guest users.

Anyway, the module is now, let's say, nearing stability. I'd like to publish a stable release soon, so let me know if you find any more bugs!

  • Like 4

Share this post


Link to post
Share on other sites

I just updated the module to version 0.5.0! I have now installed it on a couple of sites and it's pretty stable by now. But before I tag a stable 1.0.0 release I would really like to get the ProCache integration working.

ProCache integration - Testers wanted

If somebody has access to the ProCache module and would like to help me, I just need someone to test the ProCache integration!
To get started, download the procache branch of the module on Github and install it . There should be a new "ProCache" option in the module configuration. This field should also tell you whether you have ProCache installed. Check this option. Then go to the Process page through Setup -> Cache Control and click the "Clear all" button. The log output should clear the ProCache entirely. I'd like to know if you get any errors during any of those steps. Also, please check if the ProCache has been actually cleared.

Thanks!

Version 0.5.0

The new version provides some additional helper methods to check if users can access particular actions and improves the status messages displayed on the process page. In particular, there's a new static method to check if the user can access the module. This is important, because instantiating the module ($modules->get('ProcessCacheControl')) will throw an error if the user does not have the required permission. So this can now be checked with the static method ProcessCacheControl::canUseModule($user).

The complete Changelog can be found in the CHANGELOG.md in the repository.

  • Like 2

Share this post


Link to post
Share on other sites
5 hours ago, MoritzLost said:

ProCache integration - Testers wanted

I successfully tested it on one site several times (ca. 10x) with ProCaches between 50 and 700 pages! 🙂

Nice work!

  • Thanks 1

Share this post


Link to post
Share on other sites
12 hours ago, horst said:

I successfully tested it on one site several times (ca. 10x) with ProCaches between 50 and 700 pages! 🙂

Nice work!

@horst Awesome, thank you very much! I'll merge the branch into master soon then. I just need to update the documentation with all the new methods I added in the previous release, then I'll update the module to the first stable release.

I'm still thinking about other useful cache actions. I considered a "warm up" method that will just iterate over all the pages and perform anonymous HTTP requests to warm up the template render cache. Does something like this already exist? Would it be useful to anyone?
For smaller sites it would be trivial, but for larger sites the module might need some form of rate limiting, batch processing ... not sure if it's worth the effort?

  • Like 1

Share this post


Link to post
Share on other sites
59 minutes ago, MoritzLost said:

I considered a "warm up" method that will just iterate over all the pages and perform anonymous HTTP requests to warm up the template render cache. Does something like this already exist? Would it be useful to anyone?
For smaller sites it would be trivial, but for larger sites the module might need some form of rate limiting, batch processing ... not sure if it's worth the effort?

@MoritzLost A warmUp cache is definetly on the wishlist of some of us. This came up from time to time in forum posts. But with ProCache and larger sites it definetly is not trivial.

For smaller sites I have build some individual (hacky) solutions to drop and rebuild the ProCache pages, but for larger sites I manually use a local program that traverses a given website and collect information about broken links, redirects, etc.

I'm not sure how big the effort will / must be. I do know some sites (where I use my local tool) that have large archive content, that never will change. My PC time cache limit there is one year! It also could be infinite.

  • Like 2

Share this post


Link to post
Share on other sites

@horst Thanks for the feedback. Does warming up ProCache require any special consideration? I liked the approach of HTTP requests because of it's simplicity – i.e. it's virtually identical with a regular page visits, so the normal caching mechanisms can just do their thing. Or will that not work with ProCache for some reason?

Share this post


Link to post
Share on other sites

@MoritzLost I think it is "the best way!" to use (guest) HTTP requests to warm up a ProCache.  Thats why I really like the use of this crawler tool. It starts at the homepage, determine links to the homepages childpages, fetch and parse them, and so on.

I think the real problem is to get all "real public URLs" from within PW, as I have sites that use many many pages that have a public state, but in real they are only single sections of a page. So, maybe thats my private problem, but I'm not sure if there is a way to detect which pages / parent pages are official / public URLs. I think if a warmup function can get passed a optional URL list or a PageArray, that overwrites a default behavior, this would be a great benefit for all out there, that uses MVC and other individual approaches. 

  • Like 1

Share this post


Link to post
Share on other sites

@horst Hm, I hadn't considered that, that makes it more difficult indeed. Come to think of it, most of my sites would have this issue as well - for example, I use a non-standard pagination approach with URL segments, those wouldn't be caught by just using the public URL of the base page; only the first page would be reached. I guess webcrawling would be a way around that, though that is a real can of worms ...

Maybe a good compromise would be making the process hookable, like you said. Maybe the method can call a hookable method that returns an array of URLs for each page; by default it returns only the normal URLs, and additional URLs with additional parameters, url segments et c. can be added through hooks. I'll give this a go when I have the time!

  • Like 1

Share this post


Link to post
Share on other sites
1 hour ago, MoritzLost said:

though that is a real can of worms ...

😆

Share this post


Link to post
Share on other sites
8 hours ago, MoritzLost said:

I considered a "warm up" method that will just iterate over all the pages and perform anonymous HTTP requests to warm up the template render cache. Does something like this already exist? Would it be useful to anyone?

I think all caching "solutions" would somehow benefit with "warm-up" (you can try a new version of AIOM that mimics how ProCache works). Most crawlers do what you want to do more or less successfully (with some gotchas you and Horst mentioned), but it would certainly look more "professional" and it would be integrated into PW.

  • Like 2

Share this post


Link to post
Share on other sites
On 3/19/2020 at 4:09 PM, MoritzLost said:

ProCache integration - Testers wanted

@MoritzLost Nice work! Tested: the page cache is deleted, but not the compiled assets (js, css) of ProCache. PW DEV 3.0.153, ProCache latest, Win and Linux, PHP 7.3, PHP 7.4

  • Like 1

Share this post


Link to post
Share on other sites
2 hours ago, David Karich said:

@MoritzLost Nice work! Tested: the page cache is deleted, but not the compiled assets (js, css) of ProCache. PW DEV 3.0.153, ProCache latest, Win and Linux, PHP 7.3, PHP 7.4

Thanks @David Karich! Hm, right now the module is just calling $procache->clearAll() which is supposed to clear everything according to the ProCache store page. Is there another method to clear the minified assets as well? I checked all documentation I could find, a couple of ideas:

  1. I don't know how ProCache works with minified assets, but according to this blog post any changes to the CSS & JS files should be picked up automatically, so maybe it's not necessary to clear those? Though of course that is not ideal for a cache clear button 🙂
  2. Judging by the last screenshot on this blog post, ProCache stores minified assets inside /site/assets/pwpc, or is that out of date? Currently, ProcessCacheControl only supports clearing out folders inside /site/assets/cache (so it can't accidentally nuke the files directory because of a misconfigured setting ...). Maybe I could extend that to also allow clearing /site/assets/pwpc.  Is that directory path static or can it be changed?

Anyway, I'll put out a stable release with updated documentation & the ProCache integration soon! Hopefully the minification issue can also be resolved then.

  • Like 1

Share this post


Link to post
Share on other sites

I have now integrated the ProCache branch into the module and released it as version 1.0.0!

  • I have tested the module on multiple sites and it's pretty stable now. But if you do run into bugs, let me know!
  • The ProCache integration is now included, so you can clear out the ProCache page render cache alongside all other caches. Hopefully we can resolve the issue with minified assets, see the previous post.
  • The documentation is now updated with information on all the new methods added in the previous release.

Let me know if something isn't working for you!

  • Like 3

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By Macrura
      PrevNextTabs Module
      Github: https://github.com/outflux3/PrevNextTabs
      Processwire helper modules for adding page navigation within the editor.
      Overview
      This is a very simple module that adds Previous and Next links inline with the tabs on the page editor. Hovering over the tab shows the title of the previous or next page (using the admin's built in jqueryUI tooltips.)
      Usage
      This module is typically used during development where you or your editors need to traverse through pages for the purpose of proofing, flagging and/or commenting. Rather than returning to the page tree or lister, they can navigate with these links.
      Warnings
      If you are using PW version 2.6.1 or later, the system will prevent you from leaving the page if you have unsaved edits.
      For earlier versions, to avoid accidentally losing changes made to a page that might occur if a user accidentally clicks on one of these, make sure to have the Form Save Reminder module installed.
      http://modules.processwire.com/modules/prev-next-tabs/
    • By Gadgetto
      SnipWire - Snipcart integration for ProcessWire
      Snipcart is a powerful 3rd party, developer-first HTML/JavaScript shopping cart platform. SnipWire is the missing link between Snipcart and the content management framework ProcessWire.
      With SnipWire, you can quickly turn any ProcessWire site into a Snipcart online shop. The SnipWire plugin helps you to get your store up and running in no time. Detailed knowledge of the Snipcart system is not required.
      SnipWire is free and open source licensed under Mozilla Public License 2.0! A lot of work and effort has gone into development. It would be nice if you could donate an amount to support further development:

      Status update links (inside this thread) for SnipWire development
      2020-03-21 -- SnipWire 0.8.5 (beta) released! Improves SnipWires webhooks interface and provides some other fixes and additions 2020-03-03 -- SnipWire 0.8.4 (beta) released! Improves compatibility for Windows based Systems. 2020-03-01 -- SnipWire 0.8.3 (beta) released! The installation and uninstallation process has been heavily revised. 2020-02-08 -- SnipWire 0.8.2 (beta) released! Added a feature to change the cart and catalogue currency by GET, POST or SESSION param 2020-02-03 -- SnipWire 0.8.1 (beta) released! All custom classes moved into their own namespaces. 2020-02-01 -- SnipWire is now available via ProcessWire's module directory! 2020-01-30 -- SnipWire 0.8.0 (beta) first public release! (module just submitted to the PW modules directory) 2020-01-28 -- added Custom Order Fields feature (first SnipWire release version is near!) 2020-01-21 -- Snipcart v3 - when will the new cart system be implemented? 2020-01-19 -- integrated taxes provider finished (+ very flexible shipping taxes handling) 2020-01-14 -- new date range picker, discount editor, order notifiactions, order statuses, and more ... 2019-11-15 -- orders filter, order details, download + resend invoices, refunds 2019-10-18 -- list filters, REST API improvements, new docs platform, and more ... 2019-08-08 -- dashboard interface, currency selector, managing Orders, Customers and Products, Added a WireTabs, refinded caching behavior 2019-06-15 -- taxes provider, shop templates update, multiCURL implementation, and more ... 2019-06-02 -- FieldtypeSnipWireTaxSelector 2019-05-25 -- SnipWire will be free and open source Plugin Key Features
      Fast and simple store setup Full integration of the Snipcart dashboard into the ProcessWire backend (no need to leave the ProcessWire admin area) Browse and manage orders, customers, discounts, abandoned carts, and more Multi currency support Custom order and cart fields Process refunds and send customer notifications from within the ProcessWire backend Process Abandoned Carts + sending messages to customers from within the ProcessWire backend Complete Snipcart webhooks integration (all events are hookable via ProcessWire hooks) Integrated taxes provider (which is more flexible then Snipcart own provider) Useful Links
      SnipWire in PW modules directory SnipWire Docs (please note that the documentation is a work in progress) SnipWire @GitHub (feature requests and suggestions for improvement are welcome - I also accept pull requests) Snipcart Website  
      ---- INITIAL POST FROM 2019-05-25 ----
       
    • By horst
      Croppable Image 3
      for PW 3.0.20+
      Module Version 1.2.0
      Sponsored by http://dreikon.de/, many thanks Timo & Niko!
      You can get it in the modules directory!
      Please refer to the readme on github for instructions.
       
      - + - + - + - + - + - + - + - + - + - NEWS - 2020/03/19 - + - + - + - + - + - + - + - + - + - 
      There is a new Version in the pipe, that supports WebP too: 
       
      - + - + - + - + - + - + - + - + - + - NEWS - 2020/03/19 - + - + - + - + - + - + - + - + - + - 
       
       
      -------------------------------------------------------------------------
       
      Updating from prior versions:
       
      Updating from Croppable Image 3 with versions prior to 1.1.7, please do this as a one time step:
      In the PW Admin, go to side -> modules -> new, use "install via ClassName" and use CroppableImage3 for the Module Class Name. This will update your existing CroppableImage3 module sub directory, even if it is called a new install. After that, the module will be recogniced by the PW updater module, what makes it a lot easier on further updates.
      -------------------------------------------------------------------------
       
      For updating from the legacy Thumbnail / CropImage to CroppableImage3 read on here.
       
      -------------------------------------------------------------------------
       
    • By Robin S
      Inspired by a recent question.
      Image Crop Ratios
      Allows preset aspect ratios to be defined per image field for the ProcessWire image crop tool.
      The module adds a select dropdown to the crop tool. Choose an aspect ratio and the crop area will be fixed to that ratio.
      Screencast

      Installation
      Install the Image Crop Ratios module.
      Configuration
      Default aspect ratios for all image fields can be defined in the module config. Aspect ratios for specific image fields can be defined on the Input tab of the field settings. You can override the ratio settings in template context if needed. Insert a hyphen as the first item in the ratio settings unless you want to force a ratio to be applied to the crop tool. The hyphen represents a blank option that allows a free crop area to be drawn. Usage
      Click the "Crop" link on the details view of an image thumbnail. Click the "Crop" icon at the top of the editor window. Choose an option from the "Ratio" select dropdown.  
      https://github.com/Toutouwai/ImageCropRatios
      https://modules.processwire.com/modules/image-crop-ratios/
×
×
  • Create New...