Jump to content
Tyssen

Cacheing output of XML API

Recommended Posts

I'm working on a site that is using the Goodreads API but I'm finding the page very slow to render so would like to speed it up by cacheing the output of the API.

The page is made up of several child page 'bundles' each of which displays a selection of books which are chosen with a repeating field containing ISBN numbers.

$goodreads_api = 'xxxxxx';

foreach($page->children as $bundle) :

  foreach($bundle->goodreads as $goodreads) :

    $xml_string = file_get_contents('https://www.goodreads.com/search/index.xml?key='.$goodreads_api.'&q='.$goodreads->isbn);
    $book_xml = new SimpleXMLElement($xml_string);

    foreach ($book_xml->search->results->work as $book) :

    // content goes here

    endforeach;

  endforeach;

endforeach;

I've looked at the Markup Cache module but when I try cacheing the second foreach loop, I get the correct number of bundles and the correct number of books, but same output for every book.

If I cache the first foreach, I get the correct number of bundles, but only one book and again, the same content for each bundle.

So is Markup Cache the way to go, and if so, what am I doing wrong, or if not, what else should I be looking at?

Share this post


Link to post
Share on other sites

If I cache the first foreach, I get the correct number of bundles, but only one book and again, the same content for each bundle.

It would be good to see some code here, i.e. what exactly you're doing, but based on this comment it sounds like you're caching all the books with the same name and thus always overwriting previous one with the next one. Does this sound familiar?

If this is really the case, you should either generate unique name for each cached portion of the content OR cache them together instead of book-by-book.

Also: if you're using a relatively new version of ProcessWire, I'd suggest looking into the $cache API variable. MarkupCache is still valid approach, but $cache is a newer solution and widely used by the core itself. Additiionally it provides improved tools for invalidating the cache, which could come in handy here.

Share this post


Link to post
Share on other sites

The bits I left out are:

<img src="'.$book->best_book->image_url.'" alt="">
<h3 itemprop="name"><a href="https://www.goodreads.com/book/isbn/'.$goodreads->isbn.'">' . $book->best_book->title .'</a></h3>
<p class="bundle-book-author">by '. $book->best_book->author->name . '</p>

where it says // content goes here

If this is really the case, you should either generate unique name for each cached portion of the content OR cache them together instead of book-by-book.

I really just want to cache the XML output itself so that I can continue to work on the template markup and styles without having to wait several seconds for each page refresh. When you say generate a unique name for each cached portion, do you mean with a count and then something like ${'var' . $i}?

Share this post


Link to post
Share on other sites

This would cache only the xml for an hour. 

<?php

$goodreads_api = 'xxxxxx';

foreach($page->children as $bundle) :

  foreach($bundle->goodreads as $goodreads) :

    $xml_string = $cache->get($goodreads->isbn, 3600, function() use($goodreads_api, $goodreads) {
      return file_get_contents('https://www.goodreads.com/search/index.xml?key='.$goodreads_api.'&q='.$goodreads->isbn);
    });
    $book_xml = new SimpleXMLElement($xml_string);

    foreach ($book_xml->search->results->work as $book) :

    // content goes here

    endforeach;

  endforeach;

endforeach;
  • Like 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By MoritzLost
      Cacheable Placeholders
      This module allows you to have pieces of dynamic content inside cached output. This aims to solve the common problem of having a mostly cacheable site, but with pieces of dynamic output here and there.  Consider this simple example, where you want to output a custom greeting to the current user:
      <h1>Good morning, <?= ucfirst($user->name) ?></h1> This snippet means you can't use the template cache (at least for logged-in users), because each user has a different name. Even if 99% of your output is static, you can only cache the pieces that you know won't include this personal greeting. A more common example would be CSRF tokens for HTML forms - those need to be unique by definition, so you can't cache the form wholesale.
      This module solves this problem by introducing cacheable placeholders - small placeholder tokens that get replaced during every request. The replacement is done inside a Page::render hook so it runs during every request, even if the response is served from the template cache. So you can use something like this:
      <h1>Good morning, {{{greeting}}}</h1> Replacement tokens are defined with a callback function that produces the appropriate output and added to the module through a simple hook:
      // site/ready.php wire()->addHookAfter('CachePlaceholders::getTokens', function (HookEvent $e) { $tokens = $e->return; $tokens['greeting'] = [ 'callback' => function (array $tokenData) { return ucfirst(wire('user')->name); } ]; $e->return = $tokens; }); Tokens can also include parameters that are parsed and passed to the callback function. There are more fully annotated examples and step-by-step instructions in the README on Github!
      Features
      A simple and fast token parser that calls the appropriate callback and runs automatically. Tokens may include multiple named or positional parameters, as well as multi-value parameters. A manual mode that allows you to replace tokens in custom pieces of cached content (useful if you're using the $cache API). Some built-in tokens for common use-cases: CSRF-Tokens, replacing values from superglobals and producing random hexadecimal strings. The token format is completely customizable, all delimiters can be changed to avoid collisions with existing tag parsers or template languages. Links
      Github Repository & documentation Module directory If you are interested in learning more, the README is very extensive, with more usage examples, code samples and usage instructions!
    • By MoritzLost
      Process Cache Control
      This module provides a simple solution to clearing all your cache layers at once, and an extensible interface to perform various cache-related actions.
      The simple motivation behind this module was that I was tired of manually clearing caches in several places after deploying a change on a live site. The basic purpose of this module is a simple Clear all caches link in the Setup menu which clears out all caches, no matter where they hide. You can customize what exactly the module does through it's configuration menu:
      Expire or delete all cache entries in the database, or selectively clear caches by namespace ($cache API) Clear the the template render cache. Clear out specific folders inside your site's cache directory (/site/assets/cache) Clear the ProCache page render cache (if your site is using ProCache) Refresh version strings for static assets to bust client-side browser caches (this requires some setup, see the full documentation for details). This is the basic function of the module. However, you can also add different cache management action through the API and execute them through the module's interface. For this advanced usage, the module provides:
      An interface to see all available cache actions and execute them. A system log and logging output on the module page to see verify what the module is doing. A CacheControlTools class with utility functions to clear out different caches. An API to add cache actions, execute them programmatically and even modify the default action. Permission management, allowing you granular control over which user roles can execute which actions. The complete documentation can be found in the module's README.
      Plans for improvements
      If there is some interest in this, I plan to expand this to a more general cache management solution. I particular, I would like to add additional cache actions. Some ideas that came to mind:
      Warming up the template render cache for publicly accessible pages. Removing all active user sessions. Let me know if you have more suggestions!
      Links
      https://github.com/MoritzLost/ProcessCacheControl ProcessCacheControl in the Module directory CHANGELOG in the repository Screenshots


    • By Mike Rockett
      Docs & Download: rockettpw/seo/markup-sitemap
      Modules Directory: MarkupSitemap
      Composer: rockett/sitemap
      MarkupSitemap is essentially an upgrade to MarkupSitemapXML by Pete. It adds multi-language support using the built-in LanguageSupportPageNames. Where multi-language pages are available, they are added to the sitemap by means of an alternate link in that page's <url>. Support for listing images in the sitemap on a page-by-page basis and using a sitemap stylesheet are also added.
      Example when using the built-in multi-language profile:
      <url> <loc>http://domain.local/about/</loc> <lastmod>2017-08-27T16:16:32+02:00</lastmod> <xhtml:link rel="alternate" hreflang="en" href="http://domain.local/en/about/"/> <xhtml:link rel="alternate" hreflang="de" href="http://domain.local/de/uber/"/> <xhtml:link rel="alternate" hreflang="fi" href="http://domain.local/fi/tietoja/"/> </url> It also uses a locally maintained fork of a sitemap package by Matthew Davies that assists in automating the process.
      The doesn't use the same sitemap_ignore field available in MarkupSitemapXML. Rather, it renders sitemap options fields in a Page's Settings tab. One of the fields is for excluding a Page from the sitemap, and another is for excluding its children. You can assign which templates get these config fields in the module's configuration (much like you would with MarkupSEO).
      Note that the two exclusion options are mutually exclusive at this point as there may be cases where you don't want to show a parent page, but only its children. Whilst unorthodox, I'm leaving the flexibility there. (The home page cannot be excluded from the sitemap, so the applicable exclusion fields won't be available there.)
      As of December 2017, you can also exclude templates from sitemap access altogether, whilst retaining their settings if previously configured.
      Sitemap also allows you to include images for each page at the template level, and you can disable image output at the page level.
      The module allows you to set the priority on a per-page basis (it's optional and will not be included if not set).
      Lastly, a stylesheet option has also been added. You can use the default one (enabled by default), or set your own.
      Note that if the module is uninstalled, any saved data on a per-page basis is removed. The same thing happens for a specific page when it is deleted after having been trashed.
          
    • By verdeandrea
      Hello,
      I am using ProCache v3.1.8 on ProcessWire 3.0.96.
      Everything worked fine in the past, but today I noticed that the css file serverd by procache gives a 410 error. 
      The file is there, I checked.
      I deleted the cached files, I deleted the css file, I looked into the .htaccess file looking for some clues about this problem but nothing worked.
      The only way i can see my website correctly again is disabling ProCache. 
      Has anyone any clue on what could be the cause of the problem or on what should I do to fix it?
      Thanks!
    • By abdulqayyum
      Hy Processwire community,
      There are some problem in fileCompiler cache.
      when i change under the directory \site\templates\ it must change under the directory /site/assets/cache/FileCompiler/site/templates/
      but it does not update and functionality working with /site/assets/cache/FileCompiler/site/templates/ directory.
       
      In this case please suggest me how i clear fileCompiler cache?
      what i have to clear it manually?
      Thanks AbdulQayyum.
×
×
  • Create New...