ryan

Thanks that's very generous of you. If you ever find parts that aren't smart (there always are some) please let me know too. I sometimes come across my code (whether on this project or others) where I look at it and think to myself "what the hell was I thinking?". ;D I avoid coding at night for this reason.

Thank you, that's kind of you to say. But I think you are giving me too much credit. You deserve the credit here for discovering this issue here that I missed, so thanks for that.

This is one of the potential problems with using the WireSaveableItems, because the default behavior is that they load an entire table and all it's rows in memory at runtime. This is desirable for things like templates and fields because we need all this stuff in memory to build pages, and don't necessarily know what we'll need ahead of time. But for large inventories of information, there might be memory or speed issues as the number of items increases. To account for this, you can specify a selector to the load() function to tell it what you want with a selector string: $this->load($this->videosArray, "limit=10, sort=id"); That selector string can have both "start" and "limit" and "sort" options for pagination. You can also have it match any given field name in your table. For operators, you are limited to: =, <, >, !=. With those limitations in mind, they work like selectors on pages in all other respects. I also want to mention that ProcessWire does not currently use this feature of the WireSaveableItems class, so if you use it, you are the first to test it. Let me know if you run into any issues and we'll get them fixed. That's correct, but that pagination is only specific to pages. You can certainly use pagination with your module, but you'll need to specify the "start" and "limit" options to your load function manually. (When it comes to paginating actual pages, ProcessWire takes care of the "start" part for you.) When working in a database client (like PHPMyAdmin) it's true that your view of the data will be in a traditional single-table format, unlike pages where fields occupy their own table. But within ProcessWire, the view is just a manner of how you use the API code. The way you work with items coming from a WireSaveableItems class and items coming from pages is going to be the same. For example, in this part of your code: <?php $table = $this->modules->get("MarkupVideoDataTable"); $table->headerRow(array( 'Volume', 'Name', 'Description', 'Event Date', 'Price', )); foreach($videos as $video) { $volume = 'Vol. '.$video->volume; $table->row(array( $volume => "edit?id={$video->id}", $video->name, $video->description, $video->eventDate, $video->price, )); } ...$videos could just as easily be a PageArray (like from $pages->find(...) or $page->children(...), etc.), and it could be substituted without any changes. So while there is a difference in the view at the database table level, there isn't any difference at the ProcessWire API level. I don't want to talk you out of the approach you are taking, because I like what you are doing and am thrilled to see some of these classes get used outside the core -- so stick with it, and I'll be here to help. But I also want to make sure I communicate that the data you are working with is exactly what pages were designed for. Pages are data containers of a defined type (template) with defined fields... they may or may not represent an actual page on your web site, that's up to you. But pages are really useful for building huge inventories of data and making it easy to view, search and manipulate, and even change it's structure without changing your code. Think of pages like you would a node in Drupal... they may represent a page on your site, but their intended use is much more abstract than that. You can! Lets say that Users were pages. Here is how you would load an filter them by role at the same time: <?php $users = $this->pages->find("template=user, role=superuser"); foreach($users as $user) { ... } // output them in the same manner as ProcessUser.module This approach has an advantage over what's being done in ProcessUser.module right now, because it only loads the users that match the filter. The way ProcessUser works right now is that it iterates through all the users and skips the iteration if the filter doesn't match. Here's another example more specific to your case, filtering by volume number: <?php $videos = $this->pages->find("template=video, volume=123"); forech($videos as $video { ... } ...or... <?php $videos = $this->pages->get("/path/to/videos/")->children("volume=123"); foreach($video as $video) { ... } Of course, ProcessWire does this filtering at the database level (not in memory). That makes sense, and there are very good reasons for having something completely separate from a storage standpoint. From an organization standpoint, the main difference would be that the videos don't appear anywhere in the page structure. But I think your ProcessVideo.module would be equally useful whether videos are pages or WireSaveableItems (and the code behind it would likely be very much the same). Each page is assigned a template that defines it's data structure, and optionally provides a file where you can produce output for the page's URL. But that part is totally optional. If you choose to display them in some other way, the manner in which you do it should be the same. I built pages for this purpose, so that's the approach I would take myself. But I also think you have good reasons for the approach you've taken and there's nothing wrong with it. There are both tradeoffs and benefits which only you can evaluate what is best for your needs. If I'm understanding it all correctly, here are a few of the pros for your approach: 1) it can be more efficient if you are careful about how much data you load at once; 2) the data relates directly to a table, making it a simpler matter if you like to work directly in SQL; 3) it's compartmentalized, not relying upon prerequisite fieldtypes or templates (which may make for simpler portability if you distribute the module). I think in your case it is a good approach because you have PHP knowledge and know your way around code, and I am glad to see you doing it ... it also helps me to make a better product when people push the utility of these core classes. Thanks, Ryan

Hani, welcome to the forums and thanks for your interest in ProcessWire! I'm glad to see you getting down into the code. Before we take a closer look at it, I just wanted to get a better idea of the approach you've taken with extending WireSaveableItems. It's admittedly something that hadn't occurred to me to do outside of the core, as pages are the primary data type for API usage. Have you looked at using pages, templates and fields as a way to structure this data? In your case, what are the benefits to the approach you've taken? I'm actually pretty impressed with your approach, because it means you've really gotten down into the code, and you are using it like a framework. But I also worry we might be creating more work for you than is necessary by doing it that way, so I just wanted to get a better idea. ProcessWire uses WireSaveableItems for some of it's core pre-defined data types like Templates and Fields. But Users, Roles and Permissions are in the process of being converted to Page-based data types -- I want them to be things that people can easily add additional fields and data to according to the needs of their site. Page-based data types also benefit from all the page selecting, finding, pagination and manipulation functions that are built in. I haven't taken a really close look at your code yet, but looking at it so far I am wondering if you might be better of using pages, since they would translate well to your videos datatype. But I could be wrong about that, as I'm not yet an expert on what you've put together in this module, so am just posing it as a question. With regard to the error you are experiencing, I'm thinking you might need to move line 189: Wire::setFuel('videos', new Videos()); ...into your init() function. The reason is that $this->videos won't exist in your executeEdit() function. executeEdit() is only called when the URL matches "/edit/" and executeList() is only called when the URL matches "/list/" (or "/" as you've coded it.) Since you are creating the $videos object in the executeList() function, that part never gets executed in the executeEdit() function. Also, you don't even need to do Wire::setFuel('videos', new Videos()). That setFuel() method is only for things that need to be system-wide. Things like $pages, $templates, $fields, etc. It's kind of like an alternative to a global variable, but used as a way to provide the basic API objects. In your case, it looks like $videos is only used in your ProcessVideo class. So you probably only need to make $videos local to that class, rather than creating a new system-wide API variable. So I'm thinking you could put this at the top of your class: protected $videos; And then update your $videos creation line (moved to the init() function) to be this: $this->videos = new Videos(); Let me know if this makes sense, or if I'm misunderstanding? As a side note, I don't think this is what you want... but if for some reason you did want $videos to be an always-accessible system wide variable, you'd have to create it somewhere that it was automatically loaded every time ProcessWire boots... The way you'd do that is by creating a new module: createVideos.module: <?php // move your Video, VideosArray and Videos classes into this module's file. // they will be available to your Process module since this one is always loaded. class createVideos extends WireData implements Module { public static function getModuleInfo() { return array( 'title' => '...', 'version' => 100, 'summary' => '...', 'autoload' => true // tells PW to automatically instantiate this module at boot ); } public function init() { Wire::setFuel('videos', new Videos()); } } Thanks, Ryan

I was thinking about this option too, as it sounds like a good one. But then another thing occurred: what if they first uploaded hello.0x100.jpg and then later uploaded hello.jpg? Then we'd indirectly run into the same problem. I figured the safest bet is probably just to disallow uploading/adding images to PW that are in the same format that it's image cache generates, and removing any dots from the filename (other than the extension) seems to solve it. In that case, the uploaded hello.0x100.jpg becomes hello_0x100.jpg, which ProcessWire wouldn't recognize as one of it's cached images. I figured removing dots from the filename has other security benefits too, and your suggestion motivated the change. Thanks, Ryan

I installed locally too and it looks great and is very nice to use. I've setup a new admin themes page here: http://processwire.com/download/admin-themes/ I'll expand this in the future to include screenshots once we've got a couple more in there. Thanks, Ryan

The idea was that the cached filenames would be system-specific enough that it would be unlikely you'd choose the same filename as one already in the system using the same dimension format. But not so system specific that you could immediately identify what CMS it is based on the filenames. Given that, ProcessWire will assume these images are related and when the first is deleted, the other should be too: hello.jpg hello.[width]x[height].jpg While it seems like an unlikely scenario, it's not impossible that someone would upload another image to the same page called hello.123x456.jpg ... especially if they had perhaps downloaded it from another site running ProcessWire. If they did, and it matched the filename already on the given page (but with .[width]x[height].jpg appended to it) then it would get deleted when they deleted hello.jpg. That would be an error. In the latest commit, I fixed this by having ProcessWire filter out any extra dots in filenames during the upload or the $page->files->add("new file name") process. Without the extra dot there, ProcessWire isn't going to recognize it as being an image cache file, so this error should no longer be possible. Thanks for the finding this. Ryan

Thanks for finding that. I've just pushed an update to the GitHub source that fixes it.

Nice work! The screenshots look great. I can't wait to try this Monday (stuck on cell phone for the weekend). Thanks for your work with this.

It's basically a full-blown RBAC system: http://en.wikipedia.org/wiki/Role-based_access_control Despite writing the system, I still find it confusing to use in my own projects. Roles with different combinations of permissions inheriting through hierarchies are difficult to keep track of. I just want something simpler. The new system will still be role based, but you'll assign roles at the template level, and the "who can do what here" will be unambiguous. Granted the current system may be more powerful in some respects, but simplicity and ease-of-use are more important to this project. This will be for the admin. Although the result will also provide tools that can be useful on the front-end too, but there won't be any translation necessary there (since ProcessWire doesn't output anything at the front end). We definitely appreciate your help! I hope to be in touch about this within the next month or so. Thanks, Ryan

Actually this is already started. But multi language support is next on the roadmap followed by the user system. Because they depend on each other some ways, it's likely they will be built together, or close to it.

This is possible. Lets assume that you have already created your two roles. "News editor" has all the page editing permissions. And "Member" has only view permission (ProcessPageView). Edit your /news/ page and add the "News editor" role to it (from the page's settings tab). Edit your /members/ page and add the "Member" role to it. Make sure this page doesn't also have the "News editor" role. For users that you want to be able to edit /news/ and only view /members/, they should have both the "News editor" and "Member" role. Such users won't be able to edit the /members/ page(s) unless you've also turned on the "News editor" role for that page. That should achieve what you want. If it's not for some reason, check that you haven't assigned the "News editor" role to the homepage. Roles added at a lower branch are inherited through the site tree until cancelled at a higher branch. Assuming ProcessWire is working correctly, both the user and the page have to have the same role before the permissions from that role apply to the user. This is how it should work, but let me know if you find it isn't.

You shouldn't have to give edit access to the homepage in order to give edit access to a page below it. If you have a role that assigns edit access, remove that role from the homepage, and instead add it to the page where you want them to be able to edit. If you get an admin page without a list of all pages, then what's missing is the ProcessPageList permission. Make sure that is turned on for the role. ProcessPageList does not imply edit (ProcessPageEdit). So you want to give the ProcessPageList permission to all administrative roles. If a user doesn't have edit access to a page, they will still see it in the admin list (unless they don't have 'view' access), but they won't be able to edit it. Also wanted to mention that roles can be used in combination. You can assign as many roles to a user as you want, each with different permissions. They will only receive the permissions assigned to that role on pages that are also assigned that same role. It's powerful in that you have very fine grained control over access. But it's also complex and confusing, which is why I'm changing it.

The admin theme files are a combination of HTML/PHP, CSS, JS files and images. For the most part, it is no different from what you would create in your own site files (in /site/templates/). By default, the admin theme files are located in this directory: /wire/templates-admin/ STEP 1: To create a new admin theme, you would copy the default admin-templates directory (/wire/templates-admin/) to here: /site/templates-admin/ When ProcessWire sees /site/templates-admin/, it will use it instead of the default one. STEP 2: Now you can modify any of those files in /site/templates-admin/ and your changes should be reflected in the admin control panel. STEP 3: To distribute it to other people, you would just zip up the dir and then anyone else can unzip it to /site/templates-admin/ and have it work. I will be glad to setup a directory of admin themes on the site if anyone else decides to make one. That is all there is too it. But here are some other details and recommendations you may find helpful: MORE DETAILS: Why not just edit the files in /wire/templates-admin/ ? You certainly could do this, but you should consider everything in /wire/ specific to ProcessWire, and everything in /site/ specific to your installation. If you were to modify the files in /wire/templates-admin/, then your changes would get overwritten every time you upgraded ProcessWire. Whereas, /site/templates-admin/ is not overwritten during upgrades. What if I only want to change stylesheets and images? If you only need to change stylesheets and images, then I would recommend having your /site/templates-admin/default.php just include the one in /wire/templates-admin/, i.e. /site/templates/default.php: <?php include("/wire/templates-admin/default.php"); And likewise for any other files in there that have markup that you don't want to modify. This includes notices.inc and debug.inc. You'll also see a controller.php file there, but that is already just a placeholder so no need to make any changes to it. What are all the files in ProcessWire's admin theme and what do they do? Files you can edit: debug.inc - An HTML/PHP file that is included when $config->debug is true. It outputs general debugging information in the admin. default.php - The main HTML/PHP container template. All admin output goes through this file. notices.inc - Generates the markup for admin messages and error notices that appear at the top of the screen. scripts/ - Directory containing jQuery/javascript files. scripts/main.js - Main jQuery/javascript file that is run on all admin pages. All this does currently is setup a few minor form interactions. styles/ - Directory containing CSS files/stylesheets styles/main.css - Initial layout and styling for the admin control panel. styles/reset.css - Resets all browser styles styles/ui.css - User interface override styles. This is the last stylesheet loaded so it can more easily modify styles that came before it as needed. styles/JqueryUI/ Directory containing jQuery UI specific stylesheets and images. You can use jQuery UI's themeroller for these if you prefer. http://bit.ly/eTVbDC styles/ie.css - Styles specific to IE styles/ie7.css - Styles specific to IE7 and below. Because we don't support those, this basically just turns off IE7 support. styles/images/ - Images used by any of the above stylesheets. Files you shouldn't bother editing: controller.php - This is just a placeholder that includes a controller in /wire/core/. You should leave this file as-is. install-head.inc - These HTML files are used by the installer. You can edit them, but ProcessWire will still use the one in /wire/templates-admin/ since they are only used before installation. As a result, you can delete them from your /site/templates-admin/ if you prefer. install-foot.inc - See above. scripts/install.js - See above. styles/install.css - See above. Modifying form widgets with jQuery UI Most of the form inputs in ProcessWire are based on a jQuery UI theme. It is located in this file /wire/templates-admin/styles/JqueryUI/JqueryUI.css (and the images directory below it). Like the other CSS files, that can be edited directly. But since this was originally built using JQuery UI's Themeroller, you can continue to modify it with that if you'd prefer. Here is a link to it with ProcessWire's admin styles already pre-populated: http://bit.ly/eTVbDC Copyrights and Logo You should leave existing copyright notices, links to processwire.com and the name ProcessWire in place. You should also add a notice indicating "[Admin theme name] copyright [your name]", i.e. "Widget admin theme copyright Gonzo Deutschdeung". We also prefer that you keep a copy of the ProcessWire logo somewhere in your design, but it's up to you. A photoshop file of the logo at high resolution is attached to this message. pw2-logos.zip

The reason that's happening is due to a MySQL setting called ft_min_word_len. By default, MySQL doesn't index words less than 4 characters. This only affects partial match selector operators like *= and ~=. If you use an exact match selector like =, then that should still work regardless of how short the word is because it uses the exact index rather than the fulltext index. Below are instructions on how to change MySQL's ft_min_word_len setting so that you can use it with words as few as 2 characters. Before making these changes, it's good to know that this change affects all tables with fulltext indexes on the server (regardless of what database they are in). Such tables will need to be repaired after this change is made (to force them to rebuild their indexes). This is fairly simple if you have PHPMyAdmin, because you can just select all tables and choose "repair", without really worrying about whether it has a fulltext index or not. This is easy to do if you are running your own VPS or server, but may be challenging or impossible on a commercial web host (though some will have already done this). To make MySQL index words down to 2 characters, a change needs to be made in MySQL in this file: /etc/my.cnf (or wherever your mysql config file is located). Add the following to this file: [mysqld] ft_min_word_len=2 If [mysqld] is already there, then just add the ft_min_word_len=2 part. Following that, MySQL needs to be restarted, so that it reloads it's configuration file. Once restarted, everything should be back online, but you will want to run the table repair to force rebuild of indexes before your 2-letter words will actually work. REPAIR TABLE [table name] QUICK; In the case of ProcessWire, I would rebuild all the field_* tables. If you are using PhpMyAdmin, just repair all of ProcessWire's tables to make it easy. Choose the checkbox for "check all" at the bottom. Then select "Repair" in the select box to the right of it. Following that, you should be good to go with words down to 2 characters. If there are other (non ProcessWire) databases on the server, you may want to repair them too. If any are using fulltext indexes, those indexes will eventually get corrupted if not repaired. By corrupted, I don't mean data loss, I just mean that the index may not work properly until it's repaired. Let me know if I can be of any help.

Superuser is the only role that can actually edit the admin pages, and that is intentional, so that you can add stuff to it as it suits your needs. That part is hard coded, so it doesn't matter what permissions/roles you assign, it's not going to let any role other than superuser modify those pages or even see that they exist (unless you write something to do it from the API). So when you edit the /processwire/ (admin) page to add a role to it, all it's doing is giving them access to use the admin. It might say that they have edit access there, but in this instance they don't. There aren't any bugs that I'm aware of in the current user/roles system. Though definitely let me know if you found any? The main limitation is just that it's not yet designed for large scale use (i.e. lots of users or data attached to those users), and it's not as refined as other parts of the system in terms of description and messages. However, if you need to setup a role to provide edit access to just another part of the site, then the system should work quite well. This user system is going to be replaced in the near future by something that is built for lots of users and custom data attached to those users.

Hi Peter -- This has now been fixed and you should be able to install at Media Temple (and other servers that alias their document root) without having to do anything extra. I just spent some time on an MT gridserver account to make sure. PHP's realpath() function came to the rescue for this one.

Welcome to the ProcessWire forums! I'm not sure that I understand your question -- can you post more specifically? Also, the skyscraper profile is available for download. Send a request at http://ryancramer.com/contact/ and it'll send back a link to download the profile along with some information about it.

There won't be any limits, at least not any that aren't already present with pages. Though with 100k users... that's a lot. It will likely be a good test to help us introduce new performance optimizations and ways of filtering them. The existing user system was not designed to be scalable, as it was always seen as a temporary solution. Granted a temporary solution that suits the needs for most of the sites I work on. But it's not ideal if you have to deal with more than a couple dozen accounts. Sevarf2 - how many accounts are you dealing with now?

It looks to me like this problem is here: if($session->login($user, $pass)) { // login successful $session->redirect("/members/". $user); } That's where you should put your isSuperuser() check. if($session->login($user, $pass)) { // login successful if($user->isSuperuser()) $session->redirect("/processwire/"); else $session->redirect("/members/". $user); }

The current user system is bare. It's only designed to hold a few admin users. The new system will be designed for broader scope of course.

Since you are doing redirects, make sure that these are done before you output anything. Redirects use http headers, which have to come before any markup. Another thing to check is that there is no template caching being used. Check that in Admin > Templates > your template > Advanced, that the cache time is not set (or is 0). If neither of these are it, please post everything that comes before this snippet you pasted in your template (or just the whole template).

The passwords are double salted, meaning the "salt" column with each user in the DB has to go along with the password. In addition, the $config->userAuthSalt at the bottom of your /site/config.php has to match between the two sites you are trying to share the passwords with. For the users, you'll have to do this at the DB level rather than with the API. That's because ProcessWire randomizes the salt for that user every time you change the password. With the $config->userAuthSalt, you can just edit your site/config.php file to change that. Let me know if you need any help with either part. The point of all this is better security. But you can disable all of the salting if it suits your needs better by setting $config->userAuthHashType to null (which will force it to do nothing but md5 hash). You would need to do this before installation, or use the API to reset the passwords of an existing site, as any passwords already in the users table would already be salted and thus non-functional until changed.

If you are okay with the security implications of it (unencrypted or unhashed passwords), you could also just keep a copy of their current password in your /members/ pages and email it to them when they forget. You wouldn't want to do this on a site used for anything sensitive, but it may be fine to do for your needs.

I need to add some new forum sections here. I was just wondering what you all would suggest? I was thinking of possibly one or more of these, but would appreciate any suggestions. ProcessWire News + Announcements General Web Development Sites Running ProcessWire ProcessWire Tips and Tricks Bug Reports Site Optimization and Fine Tuning Core Development