How to combine Cronjob (using LazyCron) with Processwire Cache $cache?

[daniel_puehringer]

Hey community!
I would love to combine the functionalities of a Cronjob (using Lazycron Module) and the Processwire $cache API.

Idea: I want to fetch a big amount of data (HTML-Markup; many <table>-Elements) from one single website (https://www.targetwebsite.com/) at around 05:00 am each day (every 24hours) and store this fetched information for around 24hours. I then want to display this information in one template of my website (https://mywebsite.com/subpage-where-cached-data-should-be-shown) and ideally (since I bought ProCache) cache the entire Page for 24hours to be superfast.

After those 24hours a new request should fetch the data from https://targetwebsite.com/, overwrite the existing day from the day before and the whole cycle starts again.

I also thought about only using the $cache->get(...) method like this

$cachedVariable = $cache->get('foo', 86400, function() {// 86400 = 1day
	//call target website and store data directly as $cachedVariable instead of using a hook
});

However using a cronjob for this somehow seems technically better and more advanced because the first user calling the website in the morning would suffer a very long loading time.

However, if nothing works I'm thinking about calling my own website as a hook and therefore beeing the first but only user who suffers a long loading time.

Anyways, my current code snippet looks like this:

// create your hook function
function fetchDataDaily(HookEvent $e) {
    // Get an instance of WireHttp
    $cache->delete('foo');
    $http = new WireHttp();
    // Get the contents of a URL
    $response = $http->get("https://targetwebsite.com/");
    if($response !== false) {
        $cachedVariable = $cache->get('foo', 86400, function() {//86400seconds refer to 1 day
            $responseFromWebsite = $sanitizer->entities($response);
            return $responseFromWebsite;
        });
    } else {
        //echo "HTTP request failed: " . $http->getError();
    }
}
// add a hook to your function:
wire()->addHook('LazyCron::everyMinute', null, 'fetchDataDaily');

When I render the page this error is printed:
Notice: Undefined variable: cache in /html/.../site/assets/cache/FileCompiler/site/templates/home.php on line 52

 

I would really appreciate your thoughts on this - have you every had a similar problem? If so, how did you solve it? Sadly I could not find any matching posts to this problem so far.

All the best,
Daniel

[bernhard]

Hi @daniel_puehringer

What about just populating the cache via cronjob and don't care too much about the timings? The cronjob would run every 24 hours so it should be enough to just overwrite the cache when the cron runs:

// in the cronjob
$data = ...; // get data via WireHttp
$wire->cache->save('your-cache-name', $data);

// purge procache
// https://processwire.com/api/ref/pro-cache/clear-page/
$procache->clearPage($your_cached_page);

And in the template you just output that value:

echo $wire->cache->get('your-cache-name');

That should be everything you need.

To make it more robust you could create a little module that has a method that fetches data:

<?php

namespace ProcessWire;

class Site extends WireData implements Module
{

  public static function getModuleInfo()
  {
    return [
      'title' => 'Site',
      'version' => '0.0.1',
      'summary' => 'Site Module',
      'autoload' => true,
      'singular' => true,
      'icon' => 'bolt',
    ];
  }

  public function init()
  {
    // make this module available as $site API variable
    $this->wire('site', $this);
  }

  public function fetchYourData()
  {
    $data = ...; // get data via WireHttp
    $this->wire->cache->save('your-cache-name', $data);
    return $data;
  }
}

Then the cronjob gets easier:

$site->fetchYourData();
$procache->clearPage($your_cached_page);

And the template file get's more bulletproof, because if the cache is not available for whatever reason it will go and get it from the remote site:

echo $wire->cache->get('your-cache-name') ?: $site->fetchYourData();

 

[daniel_puehringer]

Servus Bernhard!
Thank you so much - that seems like a way better approach ☺️

[gebeer]

8 hours ago, daniel_puehringer said:

When I render the page this error is printed:
Notice: Undefined variable: cache in /html/.../site/assets/cache/FileCompiler/site/templates/home.php on line 52

You get this error because you are using $cache inside the hook function. There you need to use wire('cache') or $this->wire->cache, depending on the context.