Managing large sets of pages

[Michal Gancarski]

ProcessWire's default admin theme, as well as most admin themes I could find here, offers a basic tree for navigating pages. I suppose it can become quite cumbersome once the number of pages hits, say, several hundreds, which is not impossible for a e-commerce site or a blog in which each entry is a page.

How should I approach improving it? I would like to have an interface that lets me sort pages by fields in their templates for example. Should I create my own admin theme?

[kongondo]

There's a couple of ways to approach this. I can't find the forum posts atm but Ryan and others have addressed this before. One, it sometimes depends on how you have organised your pages (e.g. used sections or not?). On the other hand, some sites will have lots and lots of pages. Here, PW offers pagination. Btw, "several hundred" is not that much. Have a look at the skyscraper profile. I get your point though. However, in some cases, it does not make sense to have all your data as PW pages. This is especially in the case where those pages do not need individual URLs. For instance, a directory listing company employees. One could have this in their own database tables and have those called and displayed on one PW page. OK, back to some solutions. Have a look at this module by Soma.That can be customised to suit your needs. Going the custom database tables means you will miss out on the wonderful PW API. As for admin themes, as you know, creating your own admin theme in PW is a breeze..

[Macrura]

Soma's DataTable makes it easy to find/sort/filter large amounts of pages, but AFAIK you can't filter/sort on custom fields.

Maybe a custom admin theme or a new admin page for that purpose would be best, with an ajax search for example on the top, and sortable columns..

[diogo]

I really think the best way is to use pages. If you organize things well, those pages will never get in the way of the main structure because they will be collapsed under their parent. This is how people organize data in computers for years. The fact that you use pages will allow you to use the api, and this will make it so much easier to create a frontend or backend system to consult that data.

[aren]

For large blogs or news systems, how easy/tricky would it be to setup something like this:

Home

About

Blog

-2013

--April

--March

--February

--January

-2012

-2011

Contact

Only for organization purposes.  URLs would still appear like this: http://mysite.com/blog/my-post-title

Is it possible to setup something like this?

[kongondo]

@aren

URL segments will accomplish what you want in that case. See the following links. Also search forums for "url segments"

http://wiki.processwire.com/index.php/URL_Segments_in_category_tree_example

http://wiki.processwire.com/index.php/URL_Segments

http://processwire.com/talk/topic/3302-non-hierarchal-structure/

http://processwire.com/talk/topic/3275-hide-parent-page-from-url/

http://processwire.com/talk/topic/2822-url-segments/

Btw, maybe in future best to start own thread so that OP doesn't feel his/her post has been hijacked

[MatthewSchenker]

Greetings,

I had the same question when working on a recent project with over 500 pages (TV show listings) and growing. I am still working on this site, but my solution is JQuery DataTables. It is really simple to render your ProcessWire pages in the table, and to add sorting on any field. With just a couple more steps, you can pre-filter results to only show certain pages.

DataTables combined with ProcessWire's excellent traversal methods give you the beginnings of a very nice custom front-end interface for your clients.

I posted an example of how I am using it here: http://processwire.com/talk/topic/3128-need-guidance-for-ui-modules-sortable-data/?p=31412

JQuery DataTables is terrific for lots of other uses. Take a look at it here: http://www.datatables.net/index

There are other data table plugins, but I found DataTables to be the simplest to use, with the best documentation.

If you need to see more code, let me know.

Thanks.

Matthew

[Luis]

Im really busy these days, but fyi I setup a pw system with ~ 15.000 pages.

Trying to write a little review the next days.

[Michal Gancarski]

Thank you all for suggestions!

[Luis]

ok, a little sum up. 

To understand what happens, here is the setup:

Page called Clients, container for every client. 

every client got his unique number written to the title.

The client template itself contains a set of round about 35 fields. 

I imported a set of data via CSV import Module.

This set is provided by a client of mine who needs a new Intranet Solution.

His client Database contains exactly 15987 entries.

Every entry is one client and so every client is a page. 

 

As you can see in the screenshot, PW provides a automatic pagination. 

To handle this amount of data pagination is definitly not the best way. 

To edit and manage some clients you could use the built in search engine in the backend. 

Maybe I want to edit clients with the name Richard. 

I just pick my Clients Name field, type Richard in the Search box, et voila. 

What also could be interesting, is the performance of PW. 

Lets have a look:

<? $pages->find('template=kunde'); ?>

All clients in one PageArray gives me the following : Page generated in 33.5099 seconds.

<? $pages->find('template=kunde,limit=50'); ?>

Results in : Page generated in 0.0376 seconds.

Rendering those sets of data doesnt affect our render time much.  The following:

<? foreach($pages->find('template=kunde,limit=50') as $client): ?>
  <p>Number: <?= $client->title  ?></p>
<? endforeach ?>

Results in: Page generated in 0.0462 seconds.

Conclusion: 

At a certain point you dont have to worry about how to order pages, instead you have to worry about how to catch your pageArrays as efficient as possible. 

Managing large amounts of data is somewhat to think about at the very beginning. 

I´m handling the clients in a tree like this: Home/Database/Clients/

Hope this was helpful

[Michal Gancarski]

Luis, thank you very much! Great points, great suggestions.