Jump to content

Storing nested data


Sipho
 Share

Recommended Posts

I am trying to create a sort of database system using ProcessWire. I don't want all the data to show up as pages on the website but instead pull it into pages. Similar to the Skyscrapers demo except in that one the data shows up as pages. The data I wish to store is information about types of moths. So family, sub-family, genus and species. Each of these has information associated with it. For example, species has name, images and distribution. This is another example of information about the species. Initially, I attempted to do this using nested repeaters. This appears to work but I feel there is a better and more elegant way. Additionally, this page says: 

Quote

Repeaters aren't infinitely scalable in quantity, so avoid repeaters for quantities of items that you think may need to be infinitely scalable. 

...

It's better to use pages without repeaters when dealing with huge quantities of items. 

So I then tried using pages with children, but this also didn't work well. I had to make every single page unpublished as I don't want them showing up in menus and other places. This means they all have strikethroughs in the admin tree which is very annoying.

Is there a better way of handling this kind of data in ProcessWire or is this maybe not a suitable project for ProcessWire? I apologise if this is a silly question, but I could not find anything on this matter and am still new to ProcessWire.

Link to comment
Share on other sites

Hi @Sipho,

Welcome to the forums.

Instead of unpublished you could also make them hidden. They won't show in a $pages->find() either unless you explicit ask: $pages->find('include=hidden')'.

Also check out this famous post on how to organise content in ProcessWire with great examples. This will give you good insight on possibilities on how to structure your data.

 

  • Like 2
Link to comment
Share on other sites

In addition to all the above.

On 2017-7-10 at 8:00 PM, arjen said:

Instead of unpublished you could also make them hidden. They won't show in a $pages->find() either unless you explicit ask: $pages->find('include=hidden')'.

 @Sipho If you want to make sure hidden pages never show up accidentally (say they have a template, guest access, and someone happens to know/figure out its URL) then you can turn hidden into password protected with @adrian's great PageProtector module easily:

Another tip when dealing with related data: you might want to take a look at @Robin S's cool module:

Hope this helps too.

  • Like 2
Link to comment
Share on other sites

So after looking at 

and looking at my data again I came up with the following structure:

Capture.PNG.54fbd659269ce1a75ddd3fe6442c804d.PNG

The structure is families -> family -> subfamily -> genus -> species

This is how I originally thought of doing it, but I thought it would cause problems when I wanted to add new species and filter data. However, I have now realised that having a good page structure is very desirable. This way, people can link to the individual categories or browse through them but I can still have a separate page that references them and can filter through them.

Thanks for the help!

Also, I am loving how easy it is to use ProcessWire! It's much better than something like WordPress for a task like this. I don't have to constantly fight the system :)

 

  • Like 2
Link to comment
Share on other sites

7 minutes ago, Sipho said:

I am loving how easy it is to use ProcessWire! It's much better than something like WordPress for a task like this.

This is because as the inventor says:

"ProcessWire IS a native category system, whether by structure or relation. I think the distinction is that we don't call them "categories" or "tags". But "does not have" makes it sound like they aren't part of the plan. When in fact, it's one of the underlying purposes of the system."

to read more:

 

  • Like 2
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By prestoav
      Hi all,
      I could have sworn I used to be able to use the site generic 'title' field as a sub field of a repeater field. However I've tried to do this on two 3.0.165 sites recently and, while it will add the title field in the repeater field setup, it wont save the repeater title sub field's content when the repeater is used in a page template and edited.
      It;'s not a big issue but I wondered if this was a known restriction?
    • By David Karich
      The Page Hit Counter module for ProcessWire implements a simple page view counter in backend. Page views of visitors are automatically tracked on defined templates, with monitoring of multiple page views. This gives you a quick overview of how many visitors have read a news or a blog post, for example, without first having to open complex tools such as Google Analytics. This module quickly provides simple information, e.g. for editors. Or, for example, to sort certain news by most page views. For example for "Trending Topics".

       
      Works with ProCache and AdBlockers. With a lightweight tracking code of only ~320 bytes (gzipped). And no code changes necessary! In addition GDPR compliant, since no personal data or IP addresses are stored. Only session cookies are stored without information. 
      In addition, there are some options, for example filtering IP addresses (for CronJobs) and filtering bots, spiders and crawlers. You can also configure the lifetime of the session cookies. Repeated page views are not counted during this period. It is also possible to exclude certain roles from tracking. For example, logged in editors who work on a page are not counted as page views.

      Sort by hits and access page views (hit value)
      Each trackable template has an additional field called phits. For example, you want to output all news sorted by the number of page views.
      // It is assumed that the template, e.g. with the name "news", has been configured for tracking. $news = $pages->find("template=news, sort=-phits"); To output the page views of a tracked page, use:
      echo $page->phits; Example: Reset counter per API
      $modules->get("PageHitCounter")->resetPageViews("template=whatever", false); Example: Tracking a page hit via API and jQuery
      If you want to track a template that does not represent a full page to automatically inject a tracking script, you can define allowed API templates in the module that you can track. Below is an example of how you can track a click on news tag using jQuery. This will allow you to find out which keywords are clicked the most. For example, you can sort and display a tag cloud by the number of hits. Suppose your keywords have the template "news_tag". The template "news_tag" was also configured in the Page Hit Counter Module as a trackable API template.
      Example PHP output of keywords / tags:
      // Required: the data attribute "data-pid" with the ID of the template to be tracked. echo $pages->find("template=news_tag, sort=-phits")->each("<a href='{url}' class='news_tag' data-pid='{id}'>{title}</a>"); Example Tracking Script with jQuery:
      /** * Required: Data attribute "data-pid" with the ID of the news tag template * Required: Send the POST request to the URL "location.pathname.replace(/\/?$/, '/') + 'phcv1'" * Required: The POST parameter "pid" with the ID of the template */ $(function(){ if($('a.news_tag').length > 0) { $('a.news_tag').each(function(){ var tPID = $(this).data("pid"); if(tPID) { $(this).on("click", function(){ $.post(location.pathname.replace(/\/?$/, '/') + 'phcv1', {pid: tPID}); }); } }); } }); So simply every click on a tag is counted. Including all checks as for automatic tracking. Like Bot Filtering, Session Lifetime, etc.
      Notice: Tracking with URL segments
      If the option "Allow URL Segments" is activated on a template, the hits are only counted if the base URL of the page is called. If you want the hit to be counted even when a segment is requested, you MUST configure the segments in the template configuration. How to do this can be found here. If you use dynamic segments, configure them as RegEx. There is currently no other option. The problem is that the Page Hit Counter hooked into the PageNotFound process. If URL segments are allowed but not defined, a 404 is never triggered. This means that the Page Hit Counter cannot be called.
      New since 2.0.0: Ignore URL segments
      If a template has URL segments configured, each hit on a different segment is counted as a new hit. Enable "Ignore URL segments" so that dynamic segments are not counted individually on the base template / page.
      New since 2.0.0: Use cookieless tracking (Experimental)
      Enable this option to not use individual cookies for tracking or if you have many different pages you want to track. The limit for cookies is 50 per domain for all cookies on the page. If the option is enabled, PHP session storage is used. Downside: you can't set the lifetime higher than configured in your PHP.ini and the session will be terminated as soon as the browser is closed.
      Upgrade note for 2.0.0 from previous versions!
      Version 2.0.0 requires an update in the database schema, so that additionally the date of the last access / hit on the page can be displayed ($page->lastPageHit). To make this possible, you have to do the update via the upgrade module, upload the ZIP itself and do an update directly via the backend AND DO A MODULE REFRESH DIRECTLY AFTER UPLOAD/UPDATE. If you do not do this, you will get an error that a column is missing in the database table.
      _______________________________________________________
      Background: This module is the result of a customer requirement, where the editors are overwhelmed with analytics or no tracking tools were allowed to be used. However, a way had to be found to at least count page views in a simple form for evaluations. Furthermore, by using ProCache, a way had to be found to count views of a page without clearing the cache.
      _______________________________________________________
      Pros
      Automatic Page View Tracking Lightweight tracking code, only ~320 bytes (gzipped) No code or frontend changes necessary Works with ProCache! Even if no PHP is executed on the cached page, the tracking works Works with browser AdBlockers No cache triggers (for example, ProCache) are triggered. The cache remains persistent GDPR compliant, session-based cookie only, no personal information Filtering of IPs and bots possible Exclude certain roles from tracking Ability to reset Page Views Works with all admin themes Counter database is created as write-optimized InnoDB API to track events for templates that are not viewable No dependencies on libraries, pure VanillaJS (Automatic tracking script) Works in all modern browsers Pages are sortable by hits Cons
      Only for ProcessWire version 3.0.80 or higher (Requires wireCount()) Only for PHP version 5.6.x or higher No support for Internet Explorer <= version 9 (Because of XMLHttpRequest()) No historical data, just simple summation (Because of GDPR) Segment URLs can only be counted if the segments are defined Planned Features / ToDos
      API access to hit values Since version 1.2.1 Possibility to sort the pages by hits (Request by @Zeka) Since version 1.2.0 Don't track logged in users with certain roles (Request by @wbmnfktr) Since version 1.1.0 Possibility to reset the counter for certain pages or templates (Request by @wbmnfktr) Since version 1.1.0 Better bot filter Since version 1.1.0 Disable session lifetime, don't store cookies to track every page view (Request by @matjazp) Since version 1.2.1 Option to hide the counter in the page tree (Request by @matjazp) Since version 1.2.1 Option to hide the counter in the page tree on certain templates Since version 1.2.1 API to track events for templates that are not viewable Since version 1.2.2 Cookieless tracking Since version 2.0.0 Show last hit Since version 2.0.0 Ignore URL segments (Request by @bernhard) Since version 2.0.0 Add hookable method after pageview was tracked (Request by @bernhard) Since version 2.0.0 Changelog
       
      2.0.0
      Feature request: Add hookable method after pageview was tracked (___pageViewTracked($pageID)) (Requested by @bernhard) Feature request: Ignore URL segments option (Requested by @bernhard) New: Cookieless tracking New: Show date of last hit Update: Botlist Enhancement: Documentation improvement 1.2.7
      Feature request: make buildPageListHitCounter-Function public (Requested by @bernhard) 1.2.6
      Bug-Fix: Set the counter of a cloned page to 0 Enhancement: The function for resetting counters is now available in the module as a public function to reset counters via own scripts on the API side (Request by @VeiJari) Enhancement: Documentation improvement API reset 1.2.5
      Bug-Fix: When counting 404 hits, cookies are no longer set. The session lifetime is deactivated for the 404 page Enhancement: Documentation improvement regarding URL segments 1.2.4
      Bug-Fix: Resetting the counters on system pages (e.g. 404) does not work (Reported by wbmnfktr) Bug-Fix: Tracking endpoint is logged as 404 if module "Jumplinks" is installed (Reported by wbmnfktr) Enhancement: Corrected few typos (Merged from Sergio #6 – THX!) 1.2.3
      Bug-Fix: Tracking script triggers 404 if pages are configured without slash (#3) Reported by @maxf5 Enhancement: Reduction of the tracking script size if it's gzipped (~320 bytes) Enhancement: Documentation improvement Enhancement: Corrected few typos 1.2.2
      New feature: API to track events for templates that are not viewable Enhancement: Documentation improvement 1.2.1
      API access to hit values Use $page->phits Bug-Fix: No tracking on welcomepage (Reported by wbmnfktr; Thx to matjazp) Bug-Fix: Tracking script path on subfolders (Reported by matjazp) Bug-Fix: Tracking on pages with status "hidden" Enhancement: Change database engine to InnoDB for phits field Enhancement: Option to disable session lifetime set session lifetime to 0, no cookies Enhancement: Better installation check Enhancement: AJAX Request asyncron Enhancement: Reduction of the tracking script size by ~20% Enhancement: Option to hide the counter in the page tree You can output the counter with the field name "phits" Enhancement: Option to hide the counter in the page tree on certain templates Enhancement: Option for activate general IP validation Enhancement: Reduction of tracking overhead up to ~30ms Enhancement: Better bot list for detection 1.2.0
      New feature: Sort pages by hits – New field phits Migrate old counter data to new field 1.1.0
      New feature: Exclude tracking of certain roles New feature: Reset Page Views Better bot filter and detection 1.0.0
      Initial release Notes
      By default, the page views are stored as INT in the database. This allows a maximum counter value of 4.2 billion views (4,294,967,295) per page. If you need more, change the type to BIGINT directly in the database. But I recommend to use Google Analytics or similar tools if you have such a large number of users.
      _______________________________________________________
      Download GitHub: ProcessWire Page Hit Counter (Version 2.0.0)
      PW Module Directory: ProcessWire Page Hit Counter (Version 2.0.0)
      Install via ProcessWire (Classname): PageHitCounter
      _______________________________________________________
      Update information
      If you have used version 1.2.1 from the DEV branch, please replace it completely with the new master version.
      Old stable version
      Download GitHub: ProcessWire Page Hit Counter (Version 1.2.7)
    • By Vineet Sawant
      Hi,
      I'm trying to import some heavy data into Processwire, but I'm not sure what would be the best way to do it.
      Usually I use CSV to Pages plugin, but this time the data is too heavy(~40k rows with 10+ columns of excel sheet), thus this plugin can't help.
      I also tried Tasker plugin but I can't seem to go through the setup itself, it requires some template setup but I'm totally clueless about how to do it, so that plugin is not of any use either.
      I wanted to know from you guys how you do it and in future what would be the best way to migrate thousands of rows of data in to PW.
       
      Thanks.
       
       
    • By michelangelo
      Hello guys,
      very simple question about repeaters from the Core Modules. I have used them before without issues. This one concerns a weird behaviour which I am not sure whether it's caused by me not using it correctly or a bug. The problem is that when I add content, save it, refresh the page and return to the page, I cannot unfold the repeater. I made a video so you can see the behaviour (below)
      I tried re-initiating the repeater field, but still doesn't work...

      Looking forward to your advice!

      Thanks!


       
      repeater-problem.mp4
    • By benbyf
      Hello, and welcome to what I though was either my client being silly and changing things, or some evil doer. Turns out its reproducible and therefore something in Proceswire (I checked my templates and modules but couldnt find anything that would be doing this...). So what is it doing? Check out the video for evidence.
      A repeater field is interacting with a page template and another repeater field somehow to swap the fields in the template and repeater over...
      I have a template called team, and a repeater field called team_repeater with label Team. Some how and for some reason, when I change my fields on repeater called main_menu_links my team template gets those fields and when I try and revert the team template fields to the fields it should have, they get given to the repeater main_menu_links. Also this to say HELP!!!!!
      video: https://www.dropbox.com/s/exkdhc6n7x0xpsa/strange-repeater-PW-mega-bug.mov?dl=0
×
×
  • Create New...