New section in Admin: possible?

[Hani]

I'm new to ProcessWire - and so far I'm lovin' it.  Nice job, Ryan!

I've read through the API docs and think I have a fair understanding of the functionality.  However, I wasn't able to figure out if what I want to do is something easily done through the API, through a module, or if it has to be done outside of that architecture.

As an overview, the goal is to have a list of items that can be created, updated, deleted in the backend in the same manner as creating, updating, deleting users, roles, etc.

Basically, I created a new section in the Admin console called "Video Library".  I was able to do this by creating a new page with that name and putting it under the "Admin" page.  No problem there.

At this point, I looked at the ProcessUser and ProcessRole modules to see if I can use those modules as templates to build what I need.  I created a ProcessVideo module that extends Process. I got to a point where I can see a list of videos that are in a table in the database.  (I manually created a table called "videos" in the db and added a couple entries.) I can see it in the same format as one sees the list of users.  But when I go to edit one of the videos, I get an error that says:

Fatal error: Call to a member function get() on a non-object in T:\htdocs\shamrockvideo\site\modules\VideoLibrary\ProcessVideo.module  on line 195

That error references the following line:

if(is_null($video)) $video = isset($_GET['id']) ? $this->videos->get((int) $_GET['id']) : new Video();

I guess a definitive question to ask is: would $this->videos within my code mean I am off track?

I think I'm missing something crucial in my understanding of this all and would appreciate any guidance!  I'd really like to see this all work with ProcessWire because I think I'm falling in love with it. 

I'm attaching the module.  Please forgive me if the code is a mess and I'm not on the right track.

VideoLibrary.zip

[ryan]

Hani, welcome to the forums and thanks for your interest in ProcessWire! I'm glad to see you getting down into the code. Before we take a closer look at it, I just wanted to get a better idea of the approach you've taken with extending WireSaveableItems. It's admittedly something that hadn't occurred to me to do outside of the core, as pages are the primary data type for API usage. Have you looked at using pages, templates and fields as a way to structure this data?  In your case, what are the benefits to the approach you've taken? I'm actually pretty impressed with your approach, because it means you've really gotten down into the code, and you are using it like a framework. But I also worry we might be creating more work for you than is necessary by doing it that way, so I just wanted to get a better idea.

ProcessWire uses WireSaveableItems for some of it's core pre-defined data types like Templates and Fields.  But Users, Roles and Permissions are in the process of being converted to Page-based data types -- I want them to be things that people can easily add additional fields and data to according to the needs of their site. Page-based data types also benefit from all the page selecting, finding, pagination and manipulation functions that are built in. I haven't taken a really close look at your code yet, but looking at it so far I am wondering if you might be better of using pages, since they would translate well to your videos datatype. But I could be wrong about that, as I'm not yet an expert on what you've put together in this module, so am just posing it as a question.

With regard to the error you are experiencing, I'm thinking you might need to move line 189:

Wire::setFuel('videos', new Videos());

...into your init() function. The reason is that $this->videos won't exist in your executeEdit() function. executeEdit() is only called when the URL matches "/edit/" and executeList() is only called when the URL matches "/list/" (or "/" as you've coded it.) Since you are creating the $videos object in the executeList() function, that part never gets executed in the executeEdit() function.

Also, you don't even need to do Wire::setFuel('videos', new Videos()). That setFuel() method is only for things that need to be system-wide. Things like $pages, $templates, $fields, etc. It's kind of like an alternative to a global variable, but used as a way to provide the basic API objects. In your case, it looks like $videos is only used in your ProcessVideo class. So you probably only need to make $videos local to that class, rather than creating a new system-wide API variable. So I'm thinking you could put this at the top of your class:

protected $videos; 

And then update your $videos creation line (moved to the init() function) to be this:

$this->videos = new Videos(); 

Let me know if this makes sense, or if I'm misunderstanding?

As a side note, I don't think this is what you want... but if for some reason you did want $videos to be an always-accessible system wide variable, you'd have to create it somewhere that it was automatically loaded every time ProcessWire boots... The way you'd do that is by creating a new module:

createVideos.module:

<?php

// move your Video, VideosArray and Videos classes into this module's file.
// they will be available to your Process module since this one is always loaded.

class createVideos extends WireData implements Module {
    public static function getModuleInfo() {
        return array(
  'title' => '...',
  'version' => 100,
  'summary' => '...',
  'autoload' => true // tells PW to automatically instantiate this module at boot
          ); 
    }
    public function init() {
        Wire::setFuel('videos', new Videos()); 
    }
}

Thanks,

Ryan

[Hani]

Thanks for the super-quick and thorough reply!  I haven't had a chance to integrate your suggestions on fixing the error I'm getting, but I did want to answer your question before doing so - and see if what I'm doing really is the path to take.

You asked:

Have you looked at using pages, templates and fields as a way to structure this data?  In your case, what are the benefits to the approach you've taken?

I definitely did consider that structure the data - and it would work very easily given the architecture you built for pages.  But I guess from a pure "usability" standpoint, I'd like to take it out of that hierarchy for a couple reasons.

1. Lots of videos

I have hundreds of event videography videos/DVDs to list.  If I recall correctly, you have a pagination feature in the Pages hierarchy...so the page itself won't be miles long when viewing the children of the Video Library page.  But navigating to that "one" video to edit it wouldn't be as easy as it could be.  By giving the Video Library management page a view that is more like "Users" and "Roles", it provides more of a "table-like" data view.  Which is essentially what it is going to be.  And just like "Roles" and "Users", there will be no need to define a specific order for the videos like you do in the hierarchy of the site's pages.  (They'll be ordered by volume number.)

I guess the thing to take out of that whole blurb is that the video library is essentially a table-like data set with a lot less requirements than pages.  Each video record doesn't need to be a "standalone page"; they'll simply be listed on the website on a "Video Library" page (not to be confused with the Video Library page in the admin console).

2. Filters

The "Filter by roles" feature on the "Users" page is awesome - and I'm hoping to implement a filter by event date for the Video Library management console.  Additionally, I'm hoping to integrate a textfield in the same area where I can type in a volume number and it will filter it down to the video(s) that match.  As PW is built right now, you can't do that in Pages.

I guess for those two reasons, it feels like it might make the admin console more organized by separating Pages and the Video Library into different "data sets".

I came across a post in the forums in which someone mentioned having a "Staff" page on his website and I think he may have done it by simply adding each staff member as a page within the site's hierarchy.  That totally works and fits right in with the architecture of PW.  But I can see the method I am imagining also working for him as well - especially if the staff datatype is very specific, such as, "first name, last name, position, email, phone, photo...".  The video datatype I have is also very specific: name, description, volume, event date, price.  Granted, ALL datatypes are specific.  But this differs from the pages datatype simply by how they will be displayed on the website.

Given all that, would you approach this differently?  Or is what I'm doing seem logical?

(Just a final note that I definitely appreciate the time and effort you've put into PW! You've built an awesome product.    And I really do appreciate your time and guidance on the subject at hand as well!)

[ryan]

I have hundreds of event videography videos/DVDs to list.

This is one of the potential problems with using the WireSaveableItems, because the default behavior is that they load an entire table and all it's rows in memory at runtime. This is desirable for things like templates and fields because we need all this stuff in memory to build pages, and don't necessarily know what we'll need ahead of time. But for large inventories of information, there might be memory or speed issues as the number of items increases. To account for this, you can specify a selector to the load() function to tell it what you want with a selector string:

$this->load($this->videosArray, "limit=10, sort=id"); 

That selector string can have both "start" and "limit" and "sort" options for pagination. You can also have it match any given field name in your table. For operators, you are limited to: =, <, >, !=. With those limitations in mind, they work like selectors on pages in all other respects.

I also want to mention that ProcessWire does not currently use this feature of the WireSaveableItems class, so if you use it, you are the first to test it. Let me know if you run into any issues and we'll get them fixed.

If I recall correctly, you have a pagination feature in the Pages hierarchy...so the page itself won't be miles long when viewing the children of the Video Library page.

That's correct, but that pagination is only specific to pages. You can certainly use pagination with your module, but you'll need to specify the "start" and "limit" options to your load function manually. (When it comes to paginating actual pages, ProcessWire takes care of the "start" part for you.)

But navigating to that "one" video to edit it wouldn't be as easy as it could be.  By giving the Video Library management page a view that is more like "Users" and "Roles", it provides more of a "table-like" data view.

When working in a database client (like PHPMyAdmin) it's true that your view of the data will be in a traditional single-table format, unlike pages where fields occupy their own table. But within ProcessWire, the view is just a manner of how you use the API code. The way you work with items coming from a WireSaveableItems class and items coming from pages is going to be the same. For example, in this part of your code:

<?php
$table = $this->modules->get("MarkupVideoDataTable");
$table->headerRow(array(
  'Volume',
  'Name',
  'Description',
  'Event Date',
  'Price',
));

foreach($videos as $video) {
   $volume = 'Vol. '.$video->volume;

   $table->row(array(
   $volume => "edit?id={$video->id}",
   $video->name,
   $video->description,
   $video->eventDate,
   $video->price,
   ));
}

...$videos could just as easily be a PageArray (like from $pages->find(...) or $page->children(...), etc.), and it could be substituted without any changes. So while there is a difference in the view at the database table level, there isn't any difference at the ProcessWire API level.

I guess the thing to take out of that whole blurb is that the video library is essentially a table-like data set with a lot less requirements than pages.  Each video record doesn't need to be a "standalone page"; they'll simply be listed on the website on a "Video Library" page (not to be confused with the Video Library page in the admin console).

I don't want to talk you out of the approach you are taking, because I like what you are doing and am thrilled to see some of these classes get used outside the core -- so stick with it, and I'll be here to help. But I also want to make sure I communicate that the data you are working with is exactly what pages were designed for. Pages are data containers of a defined type (template) with defined fields... they may or may not represent an actual page on your web site, that's up to you. But pages are really useful for building huge inventories of data and making it easy to view, search and manipulate, and even change it's structure without changing your code. Think of pages like you would a node in Drupal... they may represent a page on your site, but their intended use is much more abstract than that.

The "Filter by roles" feature on the "Users" page is awesome - and I'm hoping to implement a filter by event date for the Video Library management console.  Additionally, I'm hoping to integrate a textfield in the same area where I can type in a volume number and it will filter it down to the video(s) that match.  As PW is built right now, you can't do that in Pages.

You can! Lets say that Users were pages. Here is how you would load an filter them by role at the same time:

<?php
$users = $this->pages->find("template=user, role=superuser"); 
foreach($users as $user) { ... }
// output them in the same manner as ProcessUser.module

This approach has an advantage over what's being done in ProcessUser.module right now, because it only loads the users that match the filter. The way ProcessUser works right now is that it iterates through all the users and skips the iteration if the filter doesn't match.

Here's another example more specific to your case, filtering by volume number:

<?php
$videos = $this->pages->find("template=video, volume=123"); 
forech($videos as $video { ... }

...or...

<?php
$videos = $this->pages->get("/path/to/videos/")->children("volume=123"); 
foreach($video as $video) { ... }

Of course, ProcessWire does this filtering at the database level (not in memory).

I guess for those two reasons, it feels like it might make the admin console more organized by separating Pages and the Video Library into different "data sets".

That makes sense, and there are very good reasons for having something completely separate from a storage standpoint. From an organization standpoint, the main difference would be that the videos don't appear anywhere in the page structure. But I think your ProcessVideo.module would be equally useful whether videos are pages or WireSaveableItems (and the code behind it would likely be very much the same). 

Granted, ALL datatypes are specific.  But this differs from the pages datatype simply by how they will be displayed on the website.

Each page is assigned a template that defines it's data structure, and optionally provides a file where you can produce output for the page's URL. But that part is totally optional. If you choose to display them in some other way, the manner in which you do it should be the same.

Given all that, would you approach this differently?  Or is what I'm doing seem logical?

I built pages for this purpose, so that's the approach I would take myself. But I also think you have good reasons for the approach you've taken and there's nothing wrong with it. There are both tradeoffs and benefits which only you can evaluate what is best for your needs.

If I'm understanding it all correctly, here are a few of the pros for your approach: 1) it can be more efficient if you are careful about how much data you load at once; 2) the data relates directly to a table, making it a simpler matter if you like to work directly in SQL; 3) it's compartmentalized, not relying upon prerequisite fieldtypes or templates (which may make for simpler portability if you distribute the module). I think in your case it is a good approach because you have PHP knowledge and know your way around code, and I am glad to see you doing it ... it also helps me to make a better product when people push the utility of these core classes.

Thanks,

Ryan

[Hani]

So I took some time and digested everything you said.  You indicated many drawbacks (that I wasn't aware of) concerning the method I was trying to acheive my goal.  One of the things you said was really simple, but it redirected the approach I was taking.

I don't want to talk you out of the approach you are taking, because I like what you are doing and am thrilled to see some of these classes get used outside the core -- so stick with it, and I'll be here to help.  But I also want to make sure I communicate that the data you are working with is exactly what pages were designed for.

That second sentence really stuck.  Since that is what pages were designed for, why would I want to extend the WireSaveableItems class?  I SHOULD have each of the videos be a page.  But I really would like to be able to view the videos out of the Pages hierarchy and be able to add them under a seperate section in the admin console.

It would make sense, then, to extend the classes that are more appropriate: ProcessPageAdd and Process.  (ProcessPageEdit and ProcessPageList both extend the Process class.)  By doing that I can add a new pages under Admin that execute my ProcessVideoAdd, ProcessVideoEdit, and ProcessVideoList modules.  (Actually, I don't know if you would classify them as modules since they're class extensions?  But I was able to integrate them just like modules.)

And guess what?  It worked out so well to do that!  Using those class extension/modules, markup for the list, and a "video" page template - I was able to get it working beautifully!  But now with the benefit that all the videos also show up in the site's hierarchy.  (Not that it has a direct benefit in how videos are managed - but a bonus that it jives with PW's architecture.)

I have more work to do on the ProcessVideoList module to enable pagination and filters - and I'm hoping to get to that later this evening or tomorrow.  After I do that, I'll post my work so you can check it out - and it may help someone else that's trying to do something similar.

I probably sound like a broken record every time I say, "thank you".  So I'll just tell you that you rock!    This is actually working out a lot better than I thought it would - it doesn't seem so "hackish" now!

[ryan]

It would make sense, then, to extend the classes that are more appropriate: ProcessPageAdd and Process.  (ProcessPageEdit and ProcessPageList both extend the Process class.)  By doing that I can add a new pages under Admin that execute my ProcessVideoAdd, ProcessVideoEdit, and ProcessVideoList modules.  (Actually, I don't know if you would classify them as modules since they're class extensions?  But I was able to integrate them just like modules.)

Technically modules are just a class that implements an interface. It doesn't have to extend any other class in order to implement that interface, but it does make life a little easier ... especially in the case of Process modules, because they have some flow control built in and know to look for specific CSS/JS files, etc.

There isn't any requirement that you have to use Process modules in the admin. You can just as easily use any template of your own if you prefer. But if Process modules save work for what you need to do, then I think they are a good way to go.

The ProcessPage modules are probably the broadest in scope of all of them. For most other instances, I'll usually integrate the List, Edit, Add, Delete all into a single Process module. In the case of ProcessPageEdit/Add/List, it was a lot of code so I split them up. But for your own needs, you may find you prefer keeping it to a single module, or splitting them up.. it doesn't matter. The main advantage of keeping them in a single module is that they are attached to a single page, so you don't have to worry about a multi-page dependency (not that it matters that much though).

Now that I know people are going in and using these as examples to create others, it makes me think I need to go in there and make sure all my code is clean.

And guess what?  It worked out so well to do that!  Using those class extension/modules, markup for the list, and a "video" page template - I was able to get it working beautifully!  But now with the benefit that all the videos also show up in the site's hierarchy.  (Not that it has a direct benefit in how videos are managed - but a bonus that it jives with PW's architecture.)

That was fast! Glad that it worked out well. Though I think your previous approach was a good one too. But I also think that using pages will hopefully save you time and be more fun to work with.

I'm in the process of doing the same thing that you did, but with the user system rather than videos. I'm converting the user/role/permissions system to use pages rather than WireSaveableItems, along with multi language support. From the user side, I'm doing it so that the system can accommodate any number of users (more scalable), and let you add custom fields to users, roles and permissions just like any other page template (more flexible), as well as reduce some redundancies on the multi language system. 

I have more work to do on the ProcessVideoList module to enable pagination and filters - and I'm hoping to get to that later this evening or tomorrow.  After I do that, I'll post my work so you can check it out - and it may help someone else that's trying to do something similar.

Sounds good, I look forward to seeing what you are doing. And of course, please let me know anytime I can be of any help with questions. You and a couple of other people are getting deeper into the code than what is currently in the documentation, and I'm here to help with any questions you have.

Thanks,

Ryan

[Hani]

I got really swamped, so it took me a while to finish this up.

In the case of ProcessPageEdit/Add/List, it was a lot of code so I split them up. But for your own needs, you may find you prefer keeping it to a single module, or splitting them up.. it doesn't matter. The main advantage of keeping them in a single module is that they are attached to a single page, so you don't have to worry about a multi-page dependency (not that it matters that much though).

I went ahead and finished it the way I was doing it (multiple pages) due to my lack of time and since it was pretty much done - but I love the idea of having it all attached to a single page.  When I get some time, I'm going to rework this so it uses one page instead of separate pages to list, add, and edit videos.

For anyone interested in this, attached is what I came up with.  The rundown to use this as-is:

• Put the contents of the attached zip file under modules
  
• Install all 4 modules
  
• Create a new template called "video" with all the fields you want.
  
• Add a page under the Admin section and have it run the ProcessVideoList module (also set the default child template to be "video"
  
• Add two pages under the other page you created - running either the ProcessVideoAdd and ProcessVideoEdit modules
  
• All of them use the admin template
  
• Enable page numbers for the admin template (for pagination in ProcessVideoList)
  

I think that's it!  Don't hesitate to drop me a note if you have questions about any of this!  Also, when I get some time soon, I'll actually go through the code and add comments.  I know, I know - I should have done it as I went.    Bad developer! Bad boy!

VideoLibrary.zip

[ryan]

Very cool, I look forward to checking this out. I'm wrapping up work for the day right now, so will try first thing in the morning. It sounds like you've put together a cool module here.

Put the contents of the attached zip file under modules

This may be assumed, but just wanted to mention newly added modules should always be installed in: /site/modules/ (not /wire/modules).

Install all 4 modules

Modules are self-installing, so if you have one of them that calls upon the others, then you only need to install the main one and the others will get installed automatically when called upon. Of course, there's no harm in installing them manually all at once too, but just wanted to mention this part in case it ever saves any time.

[ryan]

I can't seem to get it to install. Going through the instructions, I installed each of the modules and added the videos page in admin, then added two child pages "add" and "edit", and assigned the appropriate process. I can get to the video list page, but when I click "add video", I get:

Call to a member function getPageInputfields() on a non-object (line 40 of ProcessVideoAdd.module)

I'm wondering if I might have misunderstood part of the instructions:

Add a page under the Admin section and have it run the ProcessVideoList module (also set the default child template to be "video"

How do I set the default child template to be "video"? I think the only way is to do that is by editing an existing template... but which template?

One other thing I noticed during the install: I dragged the "VideoLibrary" dir to /site/modules/. PW could detect all the modules except for your MarkupVideoLibraryDataTable module. The reason is that it's in a dir called "Markup". Instead, the directory should be called "MarkupVideoLibraryDataTable". The reason for this is that a directory name needs to match the name of the module within it. The only exception to that is when you are creating a directory for the purpose of grouping other containing directories (like /site/modules/VideoLibrary/).

Lastly, if you are interested, the entire install process could be contained in a 1-click "install()" function in any one of your modules. For instance, an install() function in ProcessVideoList could install the other modules it needs, create the templates and pages, assign the Processes to the pages, and have everything setup with one click. This is probably worth doing if you want to create a module for distribution to others, and I'll be happy to guide you through it.

[Hani]

Oops!  I'm probably missing something critical in the instructions.  I've got a busy few days ahead of me.  After I get through them, I'll go through and set this up on a clean PW install documenting the entire procedure - and include any necessary templates. 

Sorry for the confusion!

[ryan]

Sounds good! I look forward to seeing what you come up with. Also will be having some new updates in ProcessWire over the next few weeks that I think you'll find interesting too. I'm working on multi language and user system updates, and expanding upon what you can do with using pages for data storage at the same time.