"Caching" pages field in other template

[bfncs]

Hello everyone,

I'm currently trying to find the best way to solve a problem and I'm not really sure, whether there's an easy solution.

Consider this example:

1. There is a template for "author", "article" and "tag".

2. The "article" template has a) a pages field for authors on or multiple authors and b) a pages field for on or mutilple authors tags

3. On the "authors" page I want to display a list of all tags, for which this author has articles.

The naive approach would be to just get ALL tags, iterate through them, check if there's a page with the given author and this tag and if yes, add them to the list of display tags. Something like this:

$authorTags = new PageArray();
$allTags = $pages->find('template=tag');
foreach ($allTags as $tag) {
    if($pages->count("template=article, tags=$tag")) {
        $authorTags->add($tag);
    }
}

While this works great, there's obviously a big performance problem: it doesn't scale since each additional tag results in one additional query. The cache will have to be cleared on changes of each article, which could happen pretty often, so this is not a solution.

My second approach was to have an additonal field for tag in the "author" template and write a module that hooks into page save of "article" pages, checking for changes in the tags field and replicating them in the given author's pages. In the "author" template you can then just display your tags as a "native" field, which shoud be simple and performant.

While I think, that this will probably the best and most clean solution, it will also be a lot of work, so I was wondering whether anyone of you maybe knows

a) an easier (while still performant) way to archieve this, or

b) a finished module that already solves this problem or is at least a good starting point to solve it.

Thanks a lot for your answers already,

Marc

[kongondo]

I'm trying to think about this using a music analogy.....

Article = Song

Author = Artist

Tag = Genre

In many cases, artists belong to one or two genres and their songs follow suit...

What am not clear about is are you tagging the "authors" or are you tagging the "articles". For instance, an article tag could be "health", "obesity", etc. Author tags could be "fiction", "academic", etc....

[Soma]

Not sure what's about the cache thing? Do you have cache on templates?

Well it's a naive way, but if you don't go to thousands or ten thousands of tags and articles, you should be fine. I have pretty large sites with lots of things happening like this and I don't even need caching because it's so fast.

You can always create a custom SQL query, and join the tables you need. 

Creating a module isn't complicated or lot of work at all! It same as you code in the front-end and you already know how to do it.

Here's a module done in 2 minutes and half the code is yours from above:

<?php

class TagsHelperAuthor extends WireData implements Module{

    public static function getModuleInfo(){
        return array(
         'title' => 'TagsHelperAuthor',
         'version' => 1,
         'singular' => true,
         'autoload' => true
         );
    }

    public function init(){
        $this->pages->addHookAfter("save",$this,"hookPageSave");
    }

    public function hookPageSave(HookEvent $event){
        // get current page saved
        $page = $event->argumentsByName("page");

        if($page->template == "article"){

            // save tags to page field on author page or any page
            $authorPage = wire("pages")->get("template=author, name=Authorname"); 
            $authorPage->tags->removeAll(); // remove all tags first

            $authorTags = new PageArray();
            $allTags = wire("pages")->find('template=tag');
            foreach ($allTags as $tag) {
                if(wire("pages")->count("template=article, tags=$tag")) {
                    $authorPage->tags->add($tag);
                }
            }
            $authorPage->save("tags"); // save only the field of the author page
        }
    }
}
 

Not tested but should give you a hint.

Edit: The idea is on every Article save, you update the authors "tags" field and add/save them there. You would just have to adapt it as I don't know your structure exactly.

(Changed the $pages to wire("pages") because it's in a function.)

[bfncs]

Thanks for your quick replies!

 

What am not clear about is are you tagging the "authors" or are you tagging the "articles". For instance, an article tag could be "health", "obesity", etc. Author tags could be "fiction", "academic", etc....

I have a many-to-many relation between "article" and "tag" as well as between "article" and "author". Currently this is realized with two fields for multiple page references in the "article" template.

Pages with the template "tag" are indeed something like "health", "obesity", etc., consisting only of a title field.

Pages with the template "author" represent a human being that writes articles and have a page showing a picture, the bio, etc.

[bfncs]

Took me some time to answer you, Soma, but finally:

Not sure what's about the cache thing? Do you have cache on templates?

Actually I didn't mean the usual "cache rendered page content" but only "cache indirect relations to not have to procedurally query for them on every request". Sorry for confusing here.

Well it's a naive way, but if you don't go to thousands or ten thousands of tags and articles, you should be fine. I have pretty large sites with lots of things happening like this and I don't even need caching because it's so fast.

Actually, I tried to repeat the querying and sorting with around 5 articles and 10 tags in a for loop a thousand times and took some three seconds, which is totally ok. Still I like solutions that scale better

You can always create a custom SQL query, and join the tables you need.

Well, you're definitely right here and I would probably have went for that route, but I really don't like working my way around the Pw API, so I was looking for a better solution.
 

Creating a module isn't complicated or lot of work at all! It same as you code in the front-end and you already know how to do it.
 
Here's a module done in 2 minutes and half the code is yours from above:

[...]

Edit: The idea is on every Article save, you update the authors "tags" field and add/save them there. You would just have to adapt it as I don't know your structure exactly.

Thank you so much, this is just a great and I will take that route right now! I already did quite some module programming and also even started, but my approach would have been to check only for changes on saving "articles" and add/remove from the "authors" array with that information. That would have brought up quite some edge cases to think about, which is why I posted.
You solution, on the other hand, is just doing the complete query for the "authors" tags on each save of an "article", which makes this whole thing a LOT easier. I just didn't see the wood for trees here, so thank you a lot!

Edit:

If anyone is interested in this, here is the module code I finally used, very close to what Soma posted. There are still two drawbacks with this solution:

• The querying for author's tags will still be slow. It is only done if it is necessary and it slows down the editor, not the site user, so it's ok. Might run into problems, when there are thousands of tags and/or articles.
• Currently, if you delete an author from an article, his tags aren't regenerated. This is ok for my use case (will pretty much never happen) or at least wouldn't justify the work needed to get around this limitation.

Since this is a functionality that is probably needed more often, I'll write the implementation of a universal module providing this functionality (without the two problems mentioned above for sure) on the list of "Great Processwire module ideas to do when there's enough time for it".

[Soma]

Thanks for posting your solution!

Ok got it about the cache.

Hah, yes the $array->removeAll() can be way easier than a complex add remove, possible but maybe not really needed. It's just like clear and put back the new values.

Yeah I agree with what you say. At some point you maybe better switch to an advanced SQL statement to make sure it's as fast as possible. 

Yea an universal module would be possible also but just a lot more work. But actually fun.

Something random and minor:

You can also use the shorter

$event->arguments("Page");

About the method. I was thinking if there's an simpler way to collect the tags and add them. The only one I can think of is this:

$articles = $pages->find("template=article, author=$author");
$author_tags = new PageArray();
foreach($articles as $art) $author_tags->import($art->tags);

What you think? Should be faster I think.

[bfncs]

Hey Soma, thanks for your response again!
 

Hah, yes the $array->removeAll() can be way easier than a complex add remove, possible but maybe not really needed. It's just like clear and put back the new values.

...and it totally makes sense, just something you don't see while thinking about merging differences etc ;-)

Yeah I agree with what you say. At some point you maybe better switch to an advanced SQL statement to make sure it's as fast as possible. 
 
Yea an universal module would be possible also but just a lot more work. But actually fun.

I still like the idea of a universal module much better, doing it the Processwire way (and giving something back to the incredible community. But what should I say, the list I mentioned before is pretty long and time is rare... hope to do at least some of that stuff in the next holidays.
 

Something random and minor:
 
You can also use the shorter

$event->arguments("Page");

Thanks, that's good to know. I just monkey-like copied that from your snippet.

About the method. I was thinking if there's an simpler way to collect the tags and add them. The only one I can think of is this:

$articles = $pages->find("template=article, author=$author");
$author_tags = new PageArray();
foreach($articles as $art) $author_tags->import($art->tags);

What you think? Should be faster I think.

Actually, I also thought about that but decided to go the other way. It should more or less depend on the amount of articles and tags you have.

What I expect is quite some tags but a much bigger amount of articles. So, I'm pretty sure, that your solution is a lot quicker for a low number of both because there's only one query instead of [number of tags + 1] query. When there are quite a lot of articles (say tens of thousands or so), your solution will at some point be slower than mine and has a greater risk of eventually breaking memory limits (on cheap shared hosting).

That being said... I'll probably throw your great suggestion in for now. I don't really expect this site to have tens of thousands of articles and if it ever does, I'll probably have some quite more serious problems than this, so let's give the editors the benefit of the speed increase until then.

Thanks a lot for your really great help,
Marc

Edit: Tested it, works great! I updated the gist so hopefully anyone can make use of it sometimes.

[Soma]

Yeah of course:). But you think there will be ever a author with more than some hundred articles? And what count of tags do you expect?

Nonetheless, I think if you likely to have less tags than articles this (following) will pretty fast (like what you do) and if you use markup cache to generate it only every hour or once a day:

Just to give another example, in the authors template. I'm sure you know already but just for the record.

$cache = $modules->get("MarkupCache");
// is already cached or expired?
if(!$data = $cache->get("author_tags", 3600)) {
    $tags = $pages->find("template=tags");
    foreach($tags as $tag) {
        if($pages->count("template=author, tags=$tags")) {
            $data .= "<a href='{$tag->url}'>$tag->title</a>";
        }
    }
    $cache->save($data);
}
echo $data;

[bfncs]

Yeah, I think you're pretty much right about the hundreds of articles. At least you should have the time to find an even better solution, when you're developing a site with thousands of articles.

Also thanks for the MarkupCache example. It's also really nice, but I like the module approach better in this case, because it only affects editor's performance and not page view performance.