ryan

This week we've got a really nice improvement to ProcessWire's page tree, something that I believe everyone that uses the ProcessWire admin will appreciate. Also a big focus this week was on ProDrafts and wrapping that up so that we can release it and make it available to you. More on both here: https://processwire.com/blog/posts/processwire-3.0.8-brings-smarter-faster-page-tree/

This week we have another new ProcessWire 3.x version for you that includes several updates and optimizations. We’ve also greatly expanded upon the field rendering with template files, with lots of info covered in this post. https://processwire.com/blog/posts/processwire-3.0.7-expands-field-rendering-page-path-history-and-more/

This week we’ve got some great new optimized methods added to the $pages API variable, plus full link abstraction now built-in, new sub-selector upgrades, and more! http://processwire.com/blog/posts/processwire-3.0.6-brings-pages-upgrades-and-link-abstraction/

Maybe so, that's a good idea. I'll include that in the 2017 version. Though I figure everyone already can see that, whereas a site's Google Analytics isn't public to everyone. I knew the Beer Garden required one to be a member, but had missed that there was a minimum post limit. I suppose that makes sense. Maybe rather than linking to it, I should post a screenshot in there? @Mike Rockett is that okay with you if I put a sample of the design in the post? If so, which one do you want me to use? (or maybe I link to one on your site?)

This week we're going to look at our path for growth in 2016, which is already off to a nice start! This includes where we've been, where we're going, and how we will get there–with your help. We actually do have several core updates prepared too, but mostly smaller things that will fit better in next week's post. This post originally started as a section of our 2016 roadmap, but I decided it really deserved its own post, so saved it for this week so that we could cover it more in-depth. However, you might consider this part of, or a continuation of, our 2016 roadmap. But rather than looking internally, much of this post looks externally, outside of our software and community, and into the bigger picture of ProcessWire in the web development/design world. https://processwire.com/blog/posts/growing-processwire-in-2016/

I'm guessing that some new rewrite rule the SEO guys added caused it to start using those ?it= URLs, revealing them when it usually wouldn't. For instance, maybe they setup a rewrite rule that makes /index.php redirect to "/" without getting it quite right, thereby interfering with PW's own rewrite rules. I would suggest removing whatever rule they added that's causing the interference, and then putting your own alternative in a /site/ready.php or /site/templates/_init.php (if using one), like this: if(preg_match('/(index\.php|[?&]it=/!', $_SERVER['REQUEST_URI'])) { throw new Wire404Exception(); } Or if you just want to make it redirect to the homepage, replace the exception line with $session->redirect($config->urls->root); I will add something similar to the core ProcessPageView module to prevent that possibility too.

Thanks guys, I've committed a fix for that "column not found" issue and it should now be in the module. The issue was triggered primarily by the "field_comments_votes" field, when present, as it lacks a "pages_id" column and ProcessExportProfile wasn't taking that into account. Now it does. Please let me know if you see the issue occur anymore.

@dragan, I think this is the same issue that you were seeing with ProFields (per your topic there). I think PW is having trouble understanding your paths, potentially due to the spaces present in the path names and/or something to do with your Windows environment. I've not seen any other instances of this particular issue, but I'm going to be taking a closer look at the files you sent me and hopefully we can track it down. Hopefully we can resolve it with an adjustment to FileCompiler, but I've just not been able to identify what we need to adjust yet. @tpr It's funny you should mention that because the support is already built in. I didn't document it yet because I wasn't sure I was going to keep it there. But sounds like maybe we should keep it there. The format it uses is this: field.template.php example: body.basic-page.php Since this is already built in, you can try it out now if you'd like. It checks for a template-specific file before a general purpose one. It will use the template-specific one if present, or the general purpose one if not.

This week we've released ProcessWire version 3.0.5 which actually contains quite a lot of changes. Now you can nest repeater fields (repeaters in repeaters) and use dynamic/AJAX loading for all items. Plus we've got the first test version of Repeater Matrix released, and new support for field templates… https://processwire.com/blog/posts/more-repeaters-repeater-matrix-and-new-field-rendering/

I recently installed a client site at GoDaddy (I tried to talk them out of it, but had to work with what they had). It was one of those $5/month sites, not a VPS. I had to change the RewriteBase in the .htaccess file to "/", but otherwise found that it worked just fine. Granted it's not speedy, but everything runs acceptably for a low traffic site. With ProCache installed it's actually quite fast. Based on the things you've said, I would guess that there's a bottleneck somewhere, and that the GoDaddy server is just making that bottleneck more obvious. Speed should not be significantly affected by quantity of pages in the site, so it points to something, somewhere in the site that is loading a large amount of pages (most likely a hook or module). I would enable debug mode, and then view the Debug info in the bottom of the admin screen on one of the slow loading pages (though not Lister, since it ajax-loads its pages separately). Click on the "Pages" section to see what's being loaded. I'm guessing you might have something in there loading all 2000 pages. This might help to point to the source of the bottleneck.

It's like an enhanced repeater, but it doesn't actually use multiple templates. It uses just one, and pulls a few tricks behind the scenes to make it all work, while remaining perhaps more efficient than if it were using multiple templates. One other update I've been working on this morning for regular Repeaters, and now have functional (though have not pushed to GitHub) is making all repeater items ajax-driven, so they don't load until you click on them. This enables Repeaters to scale just as well as PageTable. It's a configurable option, but one that I think will be good to enable for folks that have lots and lots of repeater items... it should make a major different in editor performance.

Thanks guys glad you like what you see! Actually I haven't done anything in that respect, other than add the option to have them be collapsed by default (something that's been requested by a few people). But I agree just having that option makes quite a difference. It is using pages in exactly the same manner as regular Repeaters. Repeater Matrix actually extends Repeater, in the PHP class sense. So the core Repeater class handles all the page management for Repeater Matrix. This also enables you to convert a Repeater to a Repeater Matrix field (or back again) just by changing the field type.

This week is all about repeaters, something that we didn't cover on the roadmap last week–we always like to save a few surprises to keep things interesting. We've also got a preview of the new Repeater Matrix field coming soon to the ProFields package… Read the post here: https://processwire.com/blog/posts/processwire-3.0.4-repeaters-revisited-preview-of-profields-matrix/

Happy New Year! Here's the ProcessWire 2016 roadmap: https://processwire.com/blog/posts/happy-new-year-heres-a-roadmap-for-processwire-in-2016/

Okay if it works with 2.7.2 then it can be made to work with PW3. I installed it to take a closer look at what the issue was, and it came down to that PW3 doesn't support compiled admin themes at all. I'd forgotten that. I've updated PW3 to support compiled admin themes that live in /site/modules/, and that's now committed to the devns branch. Testing with AdminThemeModesta, it seems to work just fine. Though there's lots missing from AdminThemeModesta in terms of current feature support, so I'm not sure I'd recommend using it, but it should work now if you want it.

@mrkhan, Modesta only shows compatibility with PW versions up to 2.4, so I'm not sure there will be an easy solution to use it with PW3 or 2.7 for that matter. There have been significant changes to admin themes between the last few 2.x versions and to my knowledge only the themes included with the core are working for 2.7 and 3.0. Though if you do manage to get it working, you may want to submit a PR to the author of the Modesta theme. I think you are on the right track with adding "namespace ProcessWire;" to the top of all the PHP files. This is something that will be necessary for an admin theme to support PW3 (unlike other modules). I think the reason you are getting the "trying to get property of non object" error is because Modesta is an admin theme module, rather than an older style admin theme. That means it would need to live in /site/modules/AdminThemeModesta/ and not in /site/templates-admin/. I can see from the error message that you've got it in /site/templates-admin/ rather than in /site/modules/. An alternative is that you may want to consider creating a custom color theme for the Reno admin, as we know the Reno admin theme supports everything in the current 2.7/3.0, but we don't know if Modesta does, even if you can get it working.

Horst, I've not been able to duplicate this one here. The code I'm using to test: <pre><?php include("./index.php"); $test = $pages->get("/about/"); echo "$test->title - $test->sort - $test->modifiedStr\n"; $test->setAndSave('sort', $test->sort + 1, array('quiet' => true)); echo "$test->title - $test->sort - $test->modifiedStr\n"; Anything I'm missing with regard to reproducing it?

Hey guys, this upgrade now appears on PW 3.0 (devns) branch. If you are using the ~= operator in a search, like "title~=disability act" it now asks MySQL for the ft_min_word_len (per @LostKobrakai's suggestion above). If any word length falls under that threshold, it delegates that word to a separate REGEXP command in MySQL that matches whole words. This is actually an improvement over the code I mentioned earlier because the previous code that converts it to use a "%=" for short words could match things like "disability fact" or "react to this disability" etc. Whereas the new core addition will only match whole words in the same way that the MySQL index would, meaning the field would only match if it contained the exact words "disability" AND "act" somewhere in the field, and not some word that happens to contain the partial word "act". To take it further, it also now uses this technique to match stopwords that MySQL doesn't full-text index. Previously it would just exclude them from the search since the presence of a stop word would prevent a search from matching. Now you can perform searches that include those stop words and match them.

SiNNuT by "make it the default" I mean "make it the default when supported". When full-text isn't available for InnoDB, we couldn't use InnoDB for things like FieldtypeText/FieldtypeTextarea. I don't plan to drop MyISAM support, just default to InnoDB in the installer when we're identified that it can be fully supported in the environment. This could be a problem with migration, like when your dev server has the needed versions and your production server doesn't, so the installer will definitely need to call more attention to these considerations when providing the db engine selection at install time. Btw, we already do use InnoDB for some tables where we want to avoid table locking (like DB sessions).

This is actually one area that I think may be too broad in scope for a tuts plus tutorial. That's because PW doesn't generate markup nor does it have an official front-end forms API, so it would be much more of an HTML and PHP tutorial than a ProcessWire one. Keep in mind the guides there are to demonstrate the simplest possible cases. We use concatenation because it's really simple and easy (and what I prefer), but that doesn't mean it's a requirement of delayed output. You could just as easily use arrays, output buffering, wireTemplateFile(), wireIncludeFile(), etc. The point is to demonstrate delaying your output until the appendTemplateFile inclusion. That's what delayed output is. ProcessWire's API is a PHP API, so I find it a lot simpler to stay in that API rather than constantly jumping in and out of HTML. After all, it requires <?php ?> tags to jump in and out of PHP from HTML, but does not require anything but quotes or heredocs to use HTML within PHP. PHP alternative syntax is handy for large blocks of conditional HTML. But for most markup generation situations I don't recommend it unless you personally find it more readable. For many of us, the additional verbosity and constant jumping in/out of PHP & HTML make it difficult to read and follow, but this is purely a preference thing as I don't think it matters to the end result at all. When it comes to writing tutorials and/or documentation, I try to communicate the concepts in the simplest/shortest way possible so that focus remains on the concept and API calls that we are trying to teach. That doesn't mean you have to do the same, or that doing it differently means you aren't adhering to the concept. Whether you are using alternative syntax or not matters little to the concept. Though since you are in "direct output" when outside of a php tag, it does make your job a little harder for delayed output. But you can amend that by isolating your markup and rendering it with wireTemplateFile(). Fantastic! @benbyf and @isellsoap just let me know if there's anything I can do to assist. Thank you for writing tutorials for PW!

er314 sorry I think I misunderstood, as I thought you were suggesting we change the behavior of the existing $pages->get() method, or make it have front-end vs. back-end state (which isn't really possible since PW's API has no front-end/back-end knowledge, and the context of the API is neither). But I'm glad you started this topic because, like I said, I think you've identified a reasonable addition to the API here that should help to further clarify the purpose of the methods for any that may have confusion. And if you and Teppo had confusion about it, then certainly others will too. We've had a $pages->findOne() method in our API since the beginning, but it's just been there as an alias to $pages->get() and just for backwards compatibility. I have dropped the purpose of that method in PW3, since we no longer need that backwards compatibility, and I very much doubt anyone uses it other than the core in a few spots (which I've updated to use the get method instead). I have changed $pages->findOne() to be an alternative to $pages->get() in PW3, that by default filters in the same way as $pages->find(). You'll see this in today's commits. I think this adds more clarity because one can now assume that any $pages method that starts with "find" is also "filtered" by default.

I have to admit I would also like to free the emojis. PW should support utf8mb4 now at install time (click the gear icon on the DB config screen). There are some hard-coded utf8 references in the install.sql files (core and site profile) but the installer updates those before executing the queries (at least it should). And any create/alter table queries that occur after that (like when adding a field, etc.) use the $config->dbCharset that is setup at install time. It sounds like we need to update some column lengths though. Going from 255 down to 250 seems fine to me (for new installs). Though maybe a little more concerned about going down to 191 for InnoDB, especially since we'll probably make InnoDB our new default before PW3 becomes the new master branch. I'll look closer to see if there are situations where the indicated columns need more than 191. In PW3, we wouldn't change the character set or DB engine of existing installations, only new installations. @Pierre-Luc I completely missed your patch before. Sorry about that, sometimes I've got information overload here and it's easy for me to miss things (I think this is likely the case for all of us). I will definitely use and apply this. Thank you for creating it. We won't need the changes to the install.sql files though, since the hard coded db charsets in the install.sql files are updated dynamically by the installer. Though I will need to double check that is working, as it's been awhile since I've tested it. @BitPoet can you expand on why you think it might be possibly dangerous? I know it would certainly be dangerous if applied to existing installations (it would delete data). But I don't think we should ever have PW modify the charset or engine of existing installations. But for new installations, do you see any potential dangers?

er314, I think your intentions are in the right place and I appreciate your determination. You are still asking for an established method with a specific purpose to have a different purpose and an access controlled front-end context, when that is not the purpose or full context of the method. So I hope you can understand why it's not realistic to just change the definition and implementation of an established method. But I get where you are coming from and think you've identified a potential helpful addition in the API, so I go ahead and add a "one" method or change the "findOne" method (in PW3) to behave as a "find just one" method with regard to access control, hidden/unpublished visibility settings, etc. While I don't personally think I will ever use this method, I think that you and Teppo have identified a couple of situations where some might find it handy. Even if one doesn't need it, perhaps just the presence of it in the API will further clarify the intentions of the methods as a whole, for folks that may have thought that the existing find() and get() were the same. I still worry a little bit about people forming an impression that PW's API is doing access control for them, when our entire API is based around providing methods for the developer to control access the way they see fit. The viewable() method is the basis of that. But we've already dipped into that territory with the find() method, and by adding the proposed method, we're not removing any control, just adding more options. So I think it's an okay. What I was stating is that I'm not aware of an instance where anyone has introduced security problems into their site as a result of using the get() method. Though you stated above that you did just that, so now I'm aware of an instance. You are correct that if a developer uses a $pages->get() method to retrieve a page that came from user input, and neglects to validate that the page is one you allow, then that could be a problem. But the same problem could surface anytime a developer neglects to properly sanitize or validate something–anything–that originated from user input. And this is not something you can skip regardless of what method you are using to retrieve a page or group of pages. Simply validating that a page is viewable does not mean it's valid for whatever operation you may be performing upon it. You would need to validate that it uses the expected template, comes from the expected parent, etc. So while we are adding a PW3 method to support your request, be careful not to get the impression that you no longer need to validate a page that originated from user input. Also keep in mind that a $pages->find() or proposed $pages->findOne() method that filters results is based on database-filtering, not runtime filtering. Part of PW's access control model supports runtime hooks to Page::viewable. If your situation includes any runtime access control options and pages you are loading as a result of user input, you shouldn't skip a $page->viewable() call regardless of what method you used to retrieve the page.

Tom, there's a setting in MySQL called ft_min_word_len and the default is 4. That means that if you have any word in your query that is less than 4 characters, MySQL won't be able to match it unless that setting has been adjusted (and you've forced it to rebuild the indexes on your db/table). That's why you aren't able to match "Disability Act", because "Act" likely falls under your ft_min_word_len. You should be able to get the result you are looking for by using %= instead. But since you want to match "disability" and "act" anywhere in your search, you'd need to do this: template=document, title%=disability, title%=act The following code should use %= for words less than 4 characters and ~= for the rest. $search = $input->get->text("search"); if($search) { $selector = ''; $parts = explode(' ', $search); foreach($parts as $key => $part) { if(strlen($part) > 3) continue; // use MySQL LIKE for words under 4 characters in length $selector .= ", title%=" . $sanitizer->selectorValue($part); unset($parts[$key]); } if(count($parts)) { // use MySQL fulltext index for words 4 characters and higher $selector .= ", title~=" . $sanitizer->selectorValue(implode(' ', $parts)); } $results = $pages->find("template=document" . $selector); } else { $error = "Please enter a search term in the search box above."; $results = null; } Come to think of it, it might be nice if PW just did this for you automatically when it detected the condition... but we don't know what the ft_min_word_len setting is.

What is front-end and what is back-end? That term is just for our point of view. The code powering the rest of PW doesn't know about this, nor does it know if you are using it for an access controlled website, or some command line API tool, or something else. While your front-end page is rendered, there might be several other core and/or 3rd party modules also doing other things (because they auto loaded, or they were requested by the page), and those modules in turn might be doing several of their own $pages API calls that have absolutely nothing to do with your output, user or access control. There is a much bigger picture than just what you are doing in your template file. The built-in filtering that you have on top of $pages->find() is actually a pretty unique case in the overall API, and the phpdoc for that method outlines it as a unique case. After all, find() is primarily used for lists and pagination. You have access to everything with your API code, but your code is not the user, and the user is not your code, so be careful not to blur those lines. The API is there to assist you in getting any content you need as quickly and easily as possible, but you are the gatekeeper. It's important to keep this in mind anywhere in the API. In the more than 5 years of this open source project, I'm not aware of any instance where someone has misused a get() method and caused a security problem. So we're talking about a hypothetical scenario of someone making a code error, that to my knowledge has never occurred before, though who knows. We could construct all kinds of scenarios where a developer could misuse the power provided and get themselves in trouble. PW is a tool that when used correctly, provides very strong security and its been my primary focus in this project. But I also don't believe in magic_quotes, safe_mode or trying to save developers from themselves by introducing obstacles. These things might seem to provide more security initially, but actually create a perception that one is no longer responsible for their own security and thus become security problems themselves. I have no qualms about an access controlled "one" function per an earlier post, which could be a useful shortcut for some developers and scenarios. But I think it's important that such a method is completely distinct and also outlined as a unique case, because PW's API context is one where the site/app developer using it carries that power and responsibility. We like to access control the user, not the developer. It's good to see this Security board finally getting some discussion.