Michael van Laar

Are there hidden scalability/performance issues when making heavy use of Repeater Matrix or PageTable fields?

Recommended Posts

Hi folks,

I have a simple(?) question, followed by a slightly lengthy background explanation:
Is it a good idea—in terms of scalability and frontend performance—to use a Repeater Matrix or PageTable field to build modular content blocks?


And here is the background explanation:

I’m quite obsessed with the idea of using modular blocks to build the content of a page. Each type of element (e. g. images, videos, quotes, code examples, but also more complex elements like image galleries, promotion boxes for products or events) gets its own set of input fields, encapsulated in a container. The page’s content—or certain parts of it—are then built by stacking containers of different element types. Using such content blocks allows all kinds of pre-processing happening behind the scenes.

By now I implemented modular content blocks in one ProcessWire instance, using a Repeater Matrix field. This works great but comes along with some serious performance issues.
Let’s ignore the backend-related performance issues. They can be solved by either not opening all Repeater items or by building a solution using a PageTable field and the PageTableExtended module instead of Repeater Matrix.

Regarding the frontend: I noticed that it’s pretty much unusable without activated cache. No surprise. This article page, for example, consists of 30 different content blocks—plus some more page references for author, category, article series, etc. Without activated cache, the loading time is simply unacceptable. The PHP object with all its sub-objects must be quite big, which explains the slow server response. With activated cache, the pages load pretty fast (letting aside the external cookie banner which is an experiment), due to the performance optimization features I built into the content block templates.

But there is more than just article pages: What about list pages? What about the XML sitemap which in my case iterates over all content blocks of all pages to include information about all images and embedded videos?

The blog where I use the Repeater Matrix content block solution is still very small (because I haven’t had the time to write articles). But what happens when such a website gets bigger? Think of a website with 150 pages, each consisting of 20 content blocks. Let’s stick to the XML sitemap example: ProcessWire would have to iterate over 3150 pages in total to render the output. This wouldn’t happen too often because the output can be saved—using the built-in cache or ProCache. But it needs to be done on a regular basis.

That’s why my question:

  • Is it a good idea to build modular content blocks like this with ProcessWire?
  • Does caching do the trick in the frontend? Or are there some hidden strings attached when a project scales?
  • Is there perhaps a better solution than using a Repeater Matrix or PageTable field?

Of course, I could turn to other CMSs which use such content blocks as part of their core concept: the PHP-based Neos or the Node.js-based ApostropheCMS , for example. (Even the plain old TYPO3 uses content blocks, but I would never voluntarily use TYPO3 if I can avoid it.) But those are far more complex to set up, implement and maintain, so I’d really like to stick to ProcessWire.

  • Like 1

Share this post


Link to post
Share on other sites

Hello @Michael van Laar,

3 hours ago, Michael van Laar said:

Is it a good idea—in terms of scalability and frontend performance—to use a Repeater Matrix or PageTable field to build modular content blocks?

To answer you main question first: Yes, I don't see a performance issue in using either.

The RepeaterMatrix and PageTable field are similar in that they use pages which can be used for content elements. I prefer the RepeaterMatrix field, because it has a better usability in that you can edit the content elements on the same page. Before there was the RepeaterMatrix field, I also used for a few times the PageTable field with the PageTableExtend module. But if you have bought the ProFields I would recommend using the RepeaterMatrix.

3 hours ago, Michael van Laar said:

Let’s ignore the backend-related performance issues. They can be solved by either not opening all Repeater items or by building a solution using a PageTable field and the PageTableExtended module instead of Repeater Matrix.

On the back-end it is best when using a RepeaterMatrix field to have the default option on, that all repeater items should be loaded via Ajax.

3 hours ago, Michael van Laar said:

This article page, for example, consists of 30 different content blocks—plus some more page references for author, category, article series, etc. Without activated cache, the loading time is simply unacceptable. The PHP object with all its sub-objects must be quite big, which explains the slow server response.

I assume the page you linked is not cached? According to WebPagetest the document was loaded after 670ms, which is good (best would be under 300ms). Everything after that are your styles and scripts which are not related to the RepeaterMatrix.

3 hours ago, Michael van Laar said:

But there is more than just article pages: What about list pages? What about the XML sitemap which in my case iterates over all content blocks of all pages to include information about all images and embedded videos?

For a regular XML sitemap, you would just output all visible pages on your website. So in your example that would be around 150 pages. But even more should be no issue. I usually use the module MarkupSitemapXML for this.

With other sitemap formats I have not much experience, I have just learned about them:

  • Image sitemap: You could output the link of a page and all it files like this.
  • Video sitemap:  This seems to work only with self hosted videos. So not embeded videos. But I could be done like mentioned before.

All of those solutions mentioned are decoupled from the RepeaterMatrix and PageTable field.

4 hours ago, Michael van Laar said:

But what happens when such a website gets bigger? Think of a website with 150 pages, each consisting of 20 content blocks. Let’s stick to the XML sitemap example: ProcessWire would have to iterate over 3150 pages in total to render the output.

On every page you only output the content elements used for the page. So the total count of content elements wouldn't matter. If you have a page with 10 or 50 content elements, it should not really be a performance issue. I can think of no use case, where you would output all content elements of all pages at the same time. As mentioned before the sitemap example can be handled without have to deal with RepeaterMatrix or PageTable.

4 hours ago, Michael van Laar said:

Is it a good idea to build modular content blocks like this with ProcessWire?

Yes, thats what the RepeaterMatrix field is build for.

4 hours ago, Michael van Laar said:

Does caching do the trick in the frontend? Or are there some hidden strings attached when a project scales?

Caching always helps if you don't have user input on your page. But depending on how you build your templates, without cache there should be no performance issue.

4 hours ago, Michael van Laar said:

Is there perhaps a better solution than using a Repeater Matrix or PageTable field?

For content elements I can't think of a better solution than RepeaterMatrix.

If you have more questions, it would be best if you would share some screenshots of your set-up. 😉

Regards, Andreas

  • Like 5

Share this post


Link to post
Share on other sites

Thanks for the answer. 🙂 So I stick with the Repeater Matrix solution.

14 hours ago, AndZyk said:

I assume the page you linked is not cached? According to WebPagetest the document was loaded after 670ms, which is good (best would be under 300ms).

The page is cached, using the default cache. The server is located in Germany. The time-to-first-bite is around 300 ms when I test it from Germany: https://www.webpagetest.org/result/181020_N8_c213855c7f91cd11779d0debf99f8db9/2/details/#waterfall_view_step1

14 hours ago, AndZyk said:

With other sitemap formats I have not much experience, I have just learned about them

I include images and videos due to SEO reasons. Because of the type of lazy loading solution I use, Google may not be able to get to know all of the images. That’s why I chose to build my own template for this instead of using the module. But of course, this is more because I wanted to test what is possible, and it is not a mandatory thing.

14 hours ago, AndZyk said:

On the back-end it is best when using a RepeaterMatrix field to have the default option on, that all repeater items should be loaded via Ajax.

Right now I still have all of the repeater items opened right away in the backend. But I use the “Inline Editor” option for CKEditor fields, so the editor button bar etc, only load when I edit the field. I like this solution because with all repeater items opened I can simply use the browsers on-page search function to get me to the section of the page I want to edit. With around 10 repeater items this works great without any performance issues. With around 30 repeater items, it gets slower, e.g. when you click somewhere into an already activated CKEditor field, it takes around 2 seconds until the cursor actually appears at this location. That’s why I thought, PageTableExtended could be a nice alternative because I could output all of the text—so it is searchable—and avoid the performance issues (because there are no directly editable fields). But I personally like Repeater MAtrix much more here. It’s much more intuitive to use for website editors.

Share this post


Link to post
Share on other sites
3 hours ago, Michael van Laar said:

The page is cached, using the default cache. The server is located in Germany. The time-to-first-bite is around 300 ms when I test it from Germany

Ah, that is a good result. Without chache would be interesting, but shouldn‘t be much higher.

3 hours ago, Michael van Laar said:

Google may not be able to get to know all of the images.

Google has no problems with indexing images that are lazyloaded in my experience. I like to lazyload them with lazysizes. But of course listing them in a special sitemap would help.😉

 

Share this post


Link to post
Share on other sites
1 hour ago, AndZyk said:

Without chache would be interesting, but shouldn‘t be much higher.

Without cache it takes about 760 ms—tested from the same location in Germany: https://www.webpagetest.org/result/181020_BK_fbeaed2dca272e2bb1aa182557e47372/1/details/#waterfall_view_step1

So there is a significant difference. Of course I could make the whole site faster by using ProCache, moving to another webspace provider and using a CDN. 😉

 

54 minutes ago, AndZyk said:

Google has no problems with indexing images that are lazyloaded in my experience. I like to lazyload them with lazysizes. But of course listing them in a special sitemap would help

1

It depends. I use lazysizes, too. And I always include a noscript version, so that all links to the various image sizes are in src attributes, so crawlers can read them. I recently read that when the Google bot crawls a page for the first time, no JavaScript is executed, so it only grabs what’s in the HTML code. After a while (hours/days/weeks, depends on the specific case) the bot returns and then takes a bit more time to execute JavaScript and does a more comprehensive crawl. So everything should work without an image sitemap.

Share this post


Link to post
Share on other sites

Did you check your site for the query count. ProcessWire generally uses quite a lot of queries and given you seem to have quite a lot of nested stuff + probably some loops over the content types it might just be that the lazy loading of fields results in your code constantly talking to the database. Adding some auto-joins here and there might already make things faster.

  • Like 2

Share this post


Link to post
Share on other sites
10 minutes ago, LostKobrakai said:

Did you check your site for the query count. ProcessWire generally uses quite a lot of queries and given you seem to have quite a lot of nested stuff + probably some loops over the content types it might just be that the lazy loading of fields results in your code constantly talking to the database. Adding some auto-joins here and there might already make things faster.

Regarding the frontend: The lazyloading is only done for images and all of the required information is already in the rendered HTML. So after the initial rendering of the page, everything is finished as far as PW is concerned.

By the way, the query count of the page with 30 content blocks is 366.

Share this post


Link to post
Share on other sites

I wasn't talking about the frontend lazy loading, but the fact that in processwire fields are lazy-loaded from the db by default. So you load a page (+ autoloaded fields; 1 query) then you access the first field in the page (1 query), then you access another field of the page (1 query) and so on. That's why autoloading fields can be so useful, as it can quite quickly drop the query count and therefore improve performance. Using fieldtype matrix makes it sounds like it's still just one page, but with 30 blocks it's more in line of loading 31 pages, the parent page and all the blocks are also pages. Now if each page lazy loads their fields you're basically in a n+1 query situation. 

  • Like 4

Share this post


Link to post
Share on other sites

I haven’t really investigated this. But it’s interesting to get a little insight into how PW handles all the database queries when rendering a page. Thanks for that. 🙂 

Perhaps I’m going to play around with the Autojoin settings a little bit. For now, caching does the job. But autoloading fields would perhaps make sense to improve the performance of the first rendering of a page, the result of which is then stored in the cache.

  • Like 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.