Jump to content
Vineet Sawant

Alternative to creating pages for temporary data storage

Recommended Posts

Hello everyone,

I'm creating a large scale application which will be used by thousands of people. Part of of the application requires storing temporary information about user's activities. For example, if user comments on certain topic, notification of same should be given to OP and people following the topic.

I thought I'd create a subpage with a repeater field having few fields like user id, timestamp & notification status, read/unread (I just don't see why using page table is better in this case, if someone can enlighten me, that'd be great.).

But I've read that every repeater item is same as new page, so I'm really confused whether I should just start keeping temporary pages or use a page with repeater field.

But for every topic, there can be hundreds of notifications, which I think will be a problematic situation in future. So I am looking for any kind of temporary storage that will not included creating pages.

Writing logs to file could be done but i'm not sure how good solution is that and how reliable it'll be.

I'd really like a good conversation with someone who understands PW, so I'm here.

Thank you all.

Share this post


Link to post
Share on other sites

Some quick comments.

Repeaters / PageTable seem like far too much overhead. 

Have you looked at the new comments system on the dev branch?

You mentioned temporary storage — do the topics only exist for a short time?

Share this post


Link to post
Share on other sites

Thanks for your reply renobird.

You mentioned temporary storage — do the topics only exist for a short time?

No, topics are permanent unless owner of the topic deletes it. The temp pages are for notifications which I want to give to the owner of the topic if someone comments or does any of other activities like adding post to favourites etc.

So basically, it's like facebook's notifications. Consider you post a photo & people are comments & liking the picture, how am I to give you notifications? I've to store all this comments/liking activity somewhere right? I've figured out a way to store comments & likes, it's just the notifications of it, which is a temporary thing, that I've to figure out.

It can be done by creating temp notification pages and add them to a notifications page array field of the post and then just check the notification status and show user the unread pages as notifications. But this way I'm wasting hell lot of page IDs. Which would be an inefficient thing to do, right?

So that's where my question comes from, how do I manage temporary storage? May be PW should provide temp pages other than usual pages with all the same features.

Sorry for the long story. :)

Share this post


Link to post
Share on other sites

Ah, Seems like the least overhead would be to create a separate table for those. The lookups would be quick, even if each topic has hundreds of notifications.

Share this post


Link to post
Share on other sites

Once each notification is marked as read, cron/lazycron could clean them out.

Share this post


Link to post
Share on other sites

I'd perhaps use WireCache for something like this, depending on how you need to retrieve these items later on.

The most obvious method would be to key it by the user who will receive the notifications - so you would create a simple array structure with the important information for the notifications, and store it using a cache name like "notifications-1142" for user ID 1142. This should make it quick to retrieve. You can then pull this out, show it, update it by modifying the notifications array, and putting it back in the cache.

Other than that - a simple module as an interface to a custom simple database table would be more than suitable.

  • Like 2

Share this post


Link to post
Share on other sites

What exactly is WireCache and where can I read more about it?

Yes, I guess I should make a table to store notifications.

Share this post


Link to post
Share on other sites

What exactly is WireCache and where can I read more about it?

Yes, I guess I should make a table to store notifications.

https://processwire.com/blog/posts/processwire-core-updates-2.5.28/#wirecache-upgrades

and a little more hot off the press:

http://processwire.com/blog/posts/processwire-2.6-rc1-2.5.29-and-more/#whats-new-this-week-processwire-2.5.29

Share this post


Link to post
Share on other sites

It's a little hard to tell exactly how you want these notifications to work but i think the relatively new System Notifications module could be a good fit. For now it is only available on the dev branch: https://github.com/ryancramerdesign/ProcessWire/tree/dev/wire/modules/System/SystemNotifications

But in the soon to be released 2.6 it will be included i think. There isn't any formal docs yet but you can study the code and read a little bit about it here: https://processwire.com/blog/posts/processwire-2.5.3-master-2.5.4-dev/#whats-new-on-the-dev-branch

Share this post


Link to post
Share on other sites

You could have a pagefield "notifications" in the users template, and create the notifications as pages connected to the topic (not one per user, just one per notification, that could be used for many users), than you would link the user to the notification, and delete it from the pagefield as soon as it is presented to him.

Does this make sense?

PS: I know you asked how to not create pages, but I think this way would at least make them much less, and easier to delete when not needed.

--

A little pseudo-code for the above:

foreach ($user->notifications as $not) {
    echo $not->body; // show the notification
$page->of(false);
    $user->notifications->remove($not); // remove it from the list
    if ( !$pages->get("template=user,notifications={$not}") ) $not->delete(); // if there are no more users to be notified, delete the page
}
$page->save("notifications"); //save the field with the notifications removed
Edited by diogo
Added the pseudo-code
  • Like 2

Share this post


Link to post
Share on other sites

Hi diogo,

I was thinking about the notifications that PW's admin theme uses. I guess you are talking about the same thing, ain't you?

I will try your suggestion on a staging pw installation. Yesterday I gave up and wrote sql queries (which I absolutely hate to do) to store all the activity in a table called activity. Then once they are shown to user, I change the activity status to read from unread.

Since it's a different table, I don't have to worry about wasting page ids.

Thank you all for your inputs.

Share this post


Link to post
Share on other sites

Uh no, diogo was talking about something different than the admin notifications. That's actually what i was talking about. PW has notices (core/notices.php) and the more advanced SystemNotifications, available in dev but uninstalled by default.

Share this post


Link to post
Share on other sites

Uh no, diogo was talking about something different than the admin notifications. That's actually what i was talking about. PW has notices (core/notices.php) and the more advanced SystemNotifications, available in dev but uninstalled by default.

Oh right, I found it. 

Does this make sense?

foreach ($user->notifications as $not) {
    echo $not->body; // show the notification
$page->of(false);
    $user->notifications->remove($not); // remove it from the list
    if ( !$pages->get("template=user,notifications={$not}") ) $not->delete(); // if there are no more users to be notified, delete the page
}
$page->save("notifications"); //save the field with the notifications removed

It makes a lot of sense now.

Share this post


Link to post
Share on other sites

Sorry, ya, that's more a frontend thing, but you could interact with the notifications system using the same method, of course.

Share this post


Link to post
Share on other sites

I implemented the SystemNotifications module on my front end, but only on the user account settings page for now to test things out..At the moment it seems to work quiet nice :)

$notices = $user->notifications;

then for example

$avatarErrorMsg = $notices->error("Your error message");
$avatarErrorMsg->expires = 30; // or however you like, doesn't even need to self expire
$notices->save(); // while writing this I'm not exactly sure if necessary
foreach($notices as $n)
{
    if($n->is("warning")) $messageClass = 'warning';
    if($n->is("error")) $messageClass = 'error';
    if($n->is("message")) $messageClass = 'success';
    $content .= "<div class='alert alert-$messageClass closable'>$n->title <i class='fa fa-close'></i></div>";
}
  • Like 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By benbyf
      Hi, Looking to create form elements on a page–some input with a colection of form inputs and the appropriate labels and variables for that input. I've used ProForms in the past and rolled out my own when creating simply one off forms, but I wonder if anyone has found a good way of allowing form creation on page editing so that clients can adhocly make and edit forms?
      Thanks
    • By David Karich
      The Page Hit Counter module for ProcessWire implements a simple page view counter in backend. Page views of visitors are automatically tracked on defined templates, with monitoring of multiple page views. This gives you a quick overview of how many visitors have read a news or a blog post, for example, without first having to open complex tools such as Google Analytics. This module quickly provides simple information, e.g. for editors. Or, for example, to sort certain news by most page views. For example for "Trending Topics".

       
      Works with ProCache and AdBlockers. With a lightweight tracking code of only ~320 bytes (gzipped). And no code changes necessary! In addition GDPR compliant, since no personal data or IP addresses are stored. Only session cookies are stored without information. 
      In addition, there are some options, for example filtering IP addresses (for CronJobs) and filtering bots, spiders and crawlers. You can also configure the lifetime of the session cookies. Repeated page views are not counted during this period. It is also possible to exclude certain roles from tracking. For example, logged in editors who work on a page are not counted as page views.

      Sort by hits and access page views (hit value)
      Each trackable template has an additional field called phits. For example, you want to output all news sorted by the number of page views.
      // It is assumed that the template, e.g. with the name "news", has been configured for tracking. $news = $pages->find("template=news, sort=-phits"); To output the page views of a tracked page, use:
      echo $page->phits; Example: Reset counter per API
      $modules->get("PageHitCounter")->resetPageViews("template=whatever", false); Example: Tracking a page hit via API and jQuery
      If you want to track a template that does not represent a full page to automatically inject a tracking script, you can define allowed API templates in the module that you can track. Below is an example of how you can track a click on news tag using jQuery. This will allow you to find out which keywords are clicked the most. For example, you can sort and display a tag cloud by the number of hits. Suppose your keywords have the template "news_tag". The template "news_tag" was also configured in the Page Hit Counter Module as a trackable API template.
      Example PHP output of keywords / tags:
      // Required: the data attribute "data-pid" with the ID of the template to be tracked. echo $pages->find("template=news_tag, sort=-phits")->each("<a href='{url}' class='news_tag' data-pid='{id}'>{title}</a>"); Example Tracking Script with jQuery:
      /** * Required: Data attribute "data-pid" with the ID of the news tag template * Required: Send the POST request to the URL "location.pathname.replace(/\/?$/, '/') + 'phcv1'" * Required: The POST parameter "pid" with the ID of the template */ $(function(){ if($('a.news_tag').length > 0) { $('a.news_tag').each(function(){ var tPID = $(this).data("pid"); if(tPID) { $(this).on("click", function(){ $.post(location.pathname.replace(/\/?$/, '/') + 'phcv1', {pid: tPID}); }); } }); } }); So simply every click on a tag is counted. Including all checks as for automatic tracking. Like Bot Filtering, Session Lifetime, etc.
      Notice: Tracking with URL segments
      If the option "Allow URL Segments" is activated on a template, the hits are only counted if the base URL of the page is called. If you want the hit to be counted even when a segment is requested, you MUST configure the segments in the template configuration. How to do this can be found here. If you use dynamic segments, configure them as RegEx. There is currently no other option. The problem is that the Page Hit Counter hooked into the PageNotFound process. If URL segments are allowed but not defined, a 404 is never triggered. This means that the Page Hit Counter cannot be called.
      _______________________________________________________
      Background: This module is the result of a customer requirement, where the editors are overwhelmed with analytics or no tracking tools were allowed to be used. However, a way had to be found to at least count page views in a simple form for evaluations. Furthermore, by using ProCache, a way had to be found to count views of a page without clearing the cache.
      _______________________________________________________
      Pros
      Automatic Page View Tracking Lightweight tracking code, only ~320 bytes (gzipped) No code or frontend changes necessary Works with ProCache! Even if no PHP is executed on the cached page, the tracking works Works with browser AdBlockers No cache triggers (for example, ProCache) are triggered. The cache remains persistent GDPR compliant, session-based cookie only, no personal information Filtering of IPs and bots possible Exclude certain roles from tracking Ability to reset Page Views Works with all admin themes Counter database is created as write-optimized InnoDB API to track events for templates that are not viewable No dependencies on libraries, pure VanillaJS (Automatic tracking script) Works in all modern browsers Pages are sortable by hits Cons
      Only for ProcessWire version 3.0.80 or higher (Requires wireCount()) Only for PHP version 5.6.x or higher No support for Internet Explorer <= version 9 (Because of XMLHttpRequest()) No historical data, just simple summation (Because of GDPR) Segment URLs can only be counted if the segments are defined Planned Features / ToDos
      API access to hit values Since version 1.2.1 Possibility to sort the pages by hits (Request by @Zeka) Since version 1.2.0 Don't track logged in users with certain roles (Request by @wbmnfktr) Since version 1.1.0 Possibility to reset the counter for certain pages or templates (Request by @wbmnfktr) Since version 1.1.0 Better bot filter Since version 1.1.0 Disable session lifetime, don't store cookies to track every page view (Request by @matjazp) Since version 1.2.1 Option to hide the counter in the page tree (Request by @matjazp) Since version 1.2.1 Option to hide the counter in the page tree on certain templates Since version 1.2.1 API to track events for templates that are not viewable Since version 1.2.2 Changelog
      1.2.6
      Bug-Fix: Set the counter of a cloned page to 0 Enhancement: The function for resetting counters is now available in the module as a public function to reset counters via own scripts on the API side (Request by @VeiJari) Enhancement: Documentation improvement API reset 1.2.5
      Bug-Fix: When counting 404 hits, cookies are no longer set. The session lifetime is deactivated for the 404 page Enhancement: Documentation improvement regarding URL segments 1.2.4
      Bug-Fix: Resetting the counters on system pages (e.g. 404) does not work (Reported by wbmnfktr) Bug-Fix: Tracking endpoint is logged as 404 if module "Jumplinks" is installed (Reported by wbmnfktr) Enhancement: Corrected few typos (Merged from Sergio #6 – THX!) 1.2.3
      Bug-Fix: Tracking script triggers 404 if pages are configured without slash (#3) Reported by @maxf5 Enhancement: Reduction of the tracking script size if it's gzipped (~320 bytes) Enhancement: Documentation improvement Enhancement: Corrected few typos 1.2.2
      New feature: API to track events for templates that are not viewable Enhancement: Documentation improvement 1.2.1
      API access to hit values Use $page->phits Bug-Fix: No tracking on welcomepage (Reported by wbmnfktr; Thx to matjazp) Bug-Fix: Tracking script path on subfolders (Reported by matjazp) Bug-Fix: Tracking on pages with status "hidden" Enhancement: Change database engine to InnoDB for phits field Enhancement: Option to disable session lifetime set session lifetime to 0, no cookies Enhancement: Better installation check Enhancement: AJAX Request asyncron Enhancement: Reduction of the tracking script size by ~20% Enhancement: Option to hide the counter in the page tree You can output the counter with the field name "phits" Enhancement: Option to hide the counter in the page tree on certain templates Enhancement: Option for activate general IP validation Enhancement: Reduction of tracking overhead up to ~30ms Enhancement: Better bot list for detection 1.2.0
      New feature: Sort pages by hits – New field phits Migrate old counter data to new field 1.1.0
      New feature: Exclude tracking of certain roles New feature: Reset Page Views Better bot filter and detection 1.0.0
      Initial release Notes
      By default, the page views are stored as INT in the database. This allows a maximum counter value of 4.2 billion views (4,294,967,295) per page. If you need more, change the type to BIGINT directly in the database. But I recommend to use Google Analytics or similar tools if you have such a large number of users.
      _______________________________________________________
      Download GitHub: ProcessWire Page Hit Counter (Version 1.2.6)
      PW Module Directory: ProcessWire Page Hit Counter (Version 1.2.6)
      Install via ProcessWire (Classname): PageHitCounter
      _______________________________________________________
      Update information
      If you have used version 1.2.1 from the DEV branch, please replace it completely with the new master version.
    • By Atlasfreeman
      Hi!
      I want to make a small site, a one page site.
      And i have this idea about doing 2 to 3 diffrent template that i can load into the index / home page.

      I want to do this with an array so that i can keep creating more topics (with the template) id needed.
      <?php include('./head.inc'); // include header markup ?> <?php $children = $page->get('template=onecolmn|twocolumn, sort=sort'); foreach($children as $child) { include($child); } ?> <?php include('./foot.inc'); // include footer markup ?>  
      As you can see i have to template wish i want to control from the backend.
      I know include does not work this way, put what is my other option to make an array that loads the whole page on an other pages.
       
       
    • By Liam88
      Hi All,
      New user over at Processwire and have been rebuilding my site based on this CMS.
      I have been able to find so many answers through Google but I'm a little stuck on this one.
      I have my services page -> services categories -> category children.
      An example of those would be - domain -> services -> ppc -> management
      I also have a set of tags which have different names - services-tag -> grow-your-traffic
      Under these tags I would have multiple links to pages such as ppc, seo, social media and so on.
      A second example would be - services-tag -> convert-your-traffic
      Under here i would have multiple links to pages such as CRO
      Now the set of tags are not visible on-site as they are only created to give overview content to the main services categories.
      Using the categories and the tags I am looking to produce a layout such as (i have also attached an image as an example:
      Tag_1 headline
      Pull all services categories linking to Tag_1
      Tag_1 snippet
      Tag_2 headline
      Pull all services categories linking to Tag_2
      Tag_2 snippet
      So far I have this snippet which is pulling in the tag content but unable to get the posts to show under each of the tags. 
      If i change the if and statement to "tags" instead of "tag" then all posts show under all tags. Where as i want it to show only the posts which are linked to that tag.
      <?php namespace ProcessWire; $tags = $pages->get("/categories-services/")->children(); // Gets the tags $posts = $pages->get("/services/")->children(); // Gets the services categories $link = $tags->ref_6; // Gets the tags and services categories link - under here you have pages_id (services cat id) and data (tags id) // Tag header and summary foreach($tags as $tag) { // This breaks down the tags into sections echo '<section id="services"> <div class="container"> <div class="row"> <h2 class="heading-1"><span>'. $tag->headline.'</span></h2> <p class="mb-5">'. $tag->summary.'</p> </div> <div class="row justify-content-around services">'; // Main services categories that link to the above tags if ($posts->id === $link->pages_id && $tag->id === $link->data){ foreach($posts as $service){ // This pulls in the services categories under the tag header. echo '<div class="card flex-card" id=""> <div class="card-img"> <a href="/'. $service->name.'" title="'. $service->name .'"> <img class="card-img-top" src="../assets/files/'. $service->id.'/'. $service->img_1.'" alt="'. $service->img_1.'" title="'. $service->img_1.'"></a> </div> <div class="card-body"> <h3 class="card-title">'. $service->headline.'</h3> <p class="card-text">'. $service->summary .'</p> <div class="card-action"> <a href="" title="'. $service->name .'" role="link" class="link">View service<span></span></a> </div> </div> </div> '; } } // Grey snippet text echo '</div> </div> </section> <div class="snip-2 light-grey"> <div class="container"> <div class="row text-center">'. $tag->get('grey') .'</div> </div> </div>'; } ?> I appreciate this is a long post but i'm trying to be clear as I appreciate everyone's time.
      Any insight into where I am going wrong is greatly appreciated.
      Liam



    • By VeiJari
      Hello, our customer doesn't like the fact that they have to first add a title and then add the other info in the form. Therefore, we've enabled the way to skip this by making a temporary file before saving.
      But the problem is that by using module schedulepages in this piece of code in the module:  
      if (!$page->isNew() && $page->publishable() && $page->isChanged('status') && $page->is(Page::statusUnpublished) && $page->publish_from) {
                  $this->session->error($this->_("“Publish From Date” field was cleared to prevent the page from being unintentionally re-published on the next Lazy Cron run."));
                  $page->publish_from = null;
                  $page->save('publish_from');
              }
      it fires even with the temporary file and therefore after first save it resets the publish_from field. Is there a way to check if template is temporary, or something along those lines?
      One solution is just to ask the customer to first save the form after adding a title, but I don't see that as a good solution.
×
×
  • Create New...