Jump to content
Tyssen

Cacheing output of XML API

Recommended Posts

I'm working on a site that is using the Goodreads API but I'm finding the page very slow to render so would like to speed it up by cacheing the output of the API.

The page is made up of several child page 'bundles' each of which displays a selection of books which are chosen with a repeating field containing ISBN numbers.

$goodreads_api = 'xxxxxx';

foreach($page->children as $bundle) :

  foreach($bundle->goodreads as $goodreads) :

    $xml_string = file_get_contents('https://www.goodreads.com/search/index.xml?key='.$goodreads_api.'&q='.$goodreads->isbn);
    $book_xml = new SimpleXMLElement($xml_string);

    foreach ($book_xml->search->results->work as $book) :

    // content goes here

    endforeach;

  endforeach;

endforeach;

I've looked at the Markup Cache module but when I try cacheing the second foreach loop, I get the correct number of bundles and the correct number of books, but same output for every book.

If I cache the first foreach, I get the correct number of bundles, but only one book and again, the same content for each bundle.

So is Markup Cache the way to go, and if so, what am I doing wrong, or if not, what else should I be looking at?

Share this post


Link to post
Share on other sites

If I cache the first foreach, I get the correct number of bundles, but only one book and again, the same content for each bundle.

It would be good to see some code here, i.e. what exactly you're doing, but based on this comment it sounds like you're caching all the books with the same name and thus always overwriting previous one with the next one. Does this sound familiar?

If this is really the case, you should either generate unique name for each cached portion of the content OR cache them together instead of book-by-book.

Also: if you're using a relatively new version of ProcessWire, I'd suggest looking into the $cache API variable. MarkupCache is still valid approach, but $cache is a newer solution and widely used by the core itself. Additiionally it provides improved tools for invalidating the cache, which could come in handy here.

Share this post


Link to post
Share on other sites

The bits I left out are:

<img src="'.$book->best_book->image_url.'" alt="">
<h3 itemprop="name"><a href="https://www.goodreads.com/book/isbn/'.$goodreads->isbn.'">' . $book->best_book->title .'</a></h3>
<p class="bundle-book-author">by '. $book->best_book->author->name . '</p>

where it says // content goes here

If this is really the case, you should either generate unique name for each cached portion of the content OR cache them together instead of book-by-book.

I really just want to cache the XML output itself so that I can continue to work on the template markup and styles without having to wait several seconds for each page refresh. When you say generate a unique name for each cached portion, do you mean with a count and then something like ${'var' . $i}?

Share this post


Link to post
Share on other sites

This would cache only the xml for an hour. 

<?php

$goodreads_api = 'xxxxxx';

foreach($page->children as $bundle) :

  foreach($bundle->goodreads as $goodreads) :

    $xml_string = $cache->get($goodreads->isbn, 3600, function() use($goodreads_api, $goodreads) {
      return file_get_contents('https://www.goodreads.com/search/index.xml?key='.$goodreads_api.'&q='.$goodreads->isbn);
    });
    $book_xml = new SimpleXMLElement($xml_string);

    foreach ($book_xml->search->results->work as $book) :

    // content goes here

    endforeach;

  endforeach;

endforeach;
  • Like 1

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By Mike Rockett
      Docs & Download: rockettpw/markup-sitemap
      Modules Directory: MarkupSitemap
      Composer: rockett/sitemap
      MarkupSitemap is essentially an upgrade to MarkupSitemapXML by Pete. It adds multi-language support using the built-in LanguageSupportPageNames. Where multi-language pages are available, they are added to the sitemap by means of an alternate link in that page's <url>. Support for listing images in the sitemap on a page-by-page basis and using a sitemap stylesheet are also added.
      Example when using the built-in multi-language profile:
      <url> <loc>http://domain.local/about/</loc> <lastmod>2017-08-27T16:16:32+02:00</lastmod> <xhtml:link rel="alternate" hreflang="en" href="http://domain.local/en/about/"/> <xhtml:link rel="alternate" hreflang="de" href="http://domain.local/de/uber/"/> <xhtml:link rel="alternate" hreflang="fi" href="http://domain.local/fi/tietoja/"/> </url> It also uses a locally maintained fork of a sitemap package by Matthew Davies that assists in automating the process.
      The doesn't use the same sitemap_ignore field available in MarkupSitemapXML. Rather, it renders sitemap options fields in a Page's Settings tab. One of the fields is for excluding a Page from the sitemap, and another is for excluding its children. You can assign which templates get these config fields in the module's configuration (much like you would with MarkupSEO).
      Note that the two exclusion options are mutually exclusive at this point as there may be cases where you don't want to show a parent page, but only its children. Whilst unorthodox, I'm leaving the flexibility there. (The home page cannot be excluded from the sitemap, so the applicable exclusion fields won't be available there.)
      As of December 2017, you can also exclude templates from sitemap access altogether, whilst retaining their settings if previously configured.
      Sitemap also allows you to include images for each page at the template level, and you can disable image output at the page level.
      The module allows you to set the priority on a per-page basis (it's optional and will not be included if not set).
      Lastly, a stylesheet option has also been added. You can use the default one (enabled by default), or set your own.
      Note that if the module is uninstalled, any saved data on a per-page basis is removed. The same thing happens for a specific page when it is deleted after having been trashed.
          
    • By stanoliver
      My aim is to output a very basic xml document which should be styled with a few css-styles.
      <?xml version = "1.0"?> <contact-info> <name>Donal Duck</name> <company>Superducks</company> <phone>(011) 123-4567</phone> </contact-info> How do I implement it with processwire?
    • By modifiedcontent
      I had upgraded my Apache configuration to include PHP7.2 and PHP7.3 for a Laravel-based script on the same server. Somehow it/I messed up a previously fine Processwire site, in a very confusing way.
      The site still looks fine, but editing template files has no effect whatsoever. It is stuck on some kind of cached version. I have already disabled PHP7's OPcache, cleared browser caches, etc, with no effect.
      The pages now apparently come from PW's assets/cache/FileCompiler folder, even though I never enabled template caching for this site.
      I have tried adding "namespace ProcessWire;" to the top of the homepage template file, but then I get this fatal error:
      My functions.php file pulls data in from another Processwire installation on the same VPS with the following line:
      $othersitedata = new ProcessWire('/home/myaccount/public_html/myothersite/site/', 'https://myothersite.com/'); That apparently still works fine; the site still displays data from the other installation, but via the "cached" template that I am now unable to change.
       
      I don't know where to start with this mess. Does any of this sound familiar to anyone? Any pointers in the right direction would be much appreciated. 
       
      Edit:
      Adding "$config->templateCompile = false;" to config.php results in the same fatal error as above. 
    • By Jan235
      Hello,
      I'm started to play around with processwire. And I like it! My local dev system is up and runnig. I'm using the template factory with Twig. Anybody who use Twig and ProCache or is it possible to use both modules?
      Thanks in advance
    • By gebeer
      Hi,
      just wanted to share something I came across while working on an import module for XML data from a web service. The XML I got was not huge, but still, loading around 3.5 MB of XML with 250+ large child nodes into memory at once with simplexml_load_file() and then looping over it had significant impact on performance.
      I searched for a solution and found this great article about how to parse large XML files.
      It basically explains how to utilize the native XMLReader class together with SimpleXMLElement to handle such situations in a memory efficient way.
      After implementing it I got a significant improval on perceived performance. No comparison in numbers to share here as I'm a bit short on time.
×
×
  • Create New...