ryan

This week most of the core dev branch commits are related to minor fixes and improvements. While last week we added a new $page->preload() method, I’m going to avoid more major additions or features so that we’re not creating more things that need lots of testing. For that reason, the commits over the next weeks or month will be similar to those from this week, so that we can get a new main/master version out as soon as possible. I was just looking at the date of our last master version (3.0.229) and see that it’s been more than a year! It feels like it’s been 3 months to me — time sure does fly! Seeing how long it’s been definitely motivates me to not wait too much longer on this next main version. The current dev branch fixes and adds quite a few things relative to 3.0.229 as well, so I think of it as being the more stable version at this point… a good sign it’s about time for a new release version. Thanks for reading and have a great weekend!

This week on the dev branch, we have a new $page->preload() method that enables you to preload multiple fields in one query (in this GitHub commit). This is kind of like what the autojoin option does (when configured with a field), or what the $pages->findJoin() method does, but with one big difference. Those options happen as part of the page loading process. Whereas $page->preload() can be applied to a page that has already loaded. Here’s one example where you might find this useful. Say you have a page living at /products/product/ and it has a hundred fields. At the top of your template file that renders the page, you could have a $page->preload(‘field1’, ‘field2’, ‘field3’); to preload all those fields before outputting them. This enables you to load field1, field2 and field3 in 1 query rather than 3. On your first call to $page->field1 it won’t trigger a load from the database and instead will return the value that has already been preloaded. You can also call $page->preload(); without any arguments, and it will preload ALL the supported fields for the page. In reality, ProcessWire is already pretty quick with loading fields, so you probably won’t benefit from preloading until the scale is quite large. While developing this, I was testing by iterating 500 pages and accessing 50 different fields from each. Without preload this took 12 seconds. With preload it took 6 seconds. So for this particular case, preloading cut the time in half. I’m not a query counter, as very often lots of simple DB queries are faster than a single big query, but I’ll mention that it also reduced the quantity of database queries by more than a factor of 10. For this large scale case, that meant more than 20000 queries down to well under 2000. Like with autojoin, there are some limitations with preloading. It supports primarily Fieldtypes that use the core loading mechanism. Some Fieldtypes (especially non-core) override this, and preload() skips over those fields. It also skips over most FieldtypeMulti (multi-row fields), but FieldtypePage is supported when used with Page fields that carry one value. Multi-value can be enabled for testing with an option you’ll find in the function $options, but like autojoin, is limited by MySQL’s group_concat limit. By default that limit is 1024, which supports 140-170 page reference values in a given page field. That's quite a lot, but I don't want to assume folks won't go over that, so it's disabled by default. I’m guessing that most won’t need the preload() function, but a few might really benefit from it, especially at larger scale. So I think it’s a worthwhile addition to the core, and another method that answers a need that may arise as an installation grows, further supporting ProcessWire’s ability to scale up as needs do. Though consider it experimental and "work in progress" at the moment, as we’ll need to do more testing to make sure it is fully stable in a broader range of situations, and further improvements are likely. Special thanks to @tuomassalo at Avoine who came up with the idea for preload() and helped me to get started developing it. Last week I told you how Pete, Jan and I met up in Holland. I also wanted to mention, a couple weeks ago, right before I left for Amsterdam, Oliver from @update AG (update.ch) in Zürich, Switzerland sent me a DM saying that he was in my neighborhood, so we got together for coffee near my house. They’ve been using ProcessWire at Update AG almost as long as PW has been open source, so it was good to meet another long time ProcessWire user, and a nice coincidence that he was in the neighborhood. It’s always great to meet ProcessWire users in person and I hope to meet more of you in the future as well. Thank you for reading and have a great weekend!

The traveling over the last month or so is finally finished. In late September/early October my family traveled to Spain, France, and Italy for the first time. And the last couple weeks my wife and I were in Holland on a bike trip where we lived on a boat for a week and biked all over the Netherlands (~150 miles of biking), and got to see a large portion of it. Our forum administrator @Pete was also there, as was Jan, who maintains our website on AWS, so sometimes it felt like a mini ProcessWire meetup too. The trip was one from Tripsite, a company using ProcessWire for more than 15 years, and this trip was their 25th anniversary. There were about 30 other people there as well, several whom also work ProcessWire as editors. It was an amazing trip, and now I'm completely sold on bike and boat trips being the best way to experience a country. I felt like I was a resident rather than a tourist. I’m sorry there have not been a lot of updates here lately due to all of the travel, but now that it’s done, it’s time to get back to work on our next main/master version, which I’m greatly looking forward to. While there have only been 3 commits this week, there have been 25 commits since 3.0.241, so I’m bumping the dev branch version up to 3.0.242, to get the momentum going again. Thanks for reading, and for your patience while I catch up with communications and such, and have a great weekend! Below is a photo of Pete, Jan and Ryan on the boat in Amsterdam.

@cpx3 Thanks! It should be fixed now, but let me know if you still can't access.

@cpx3 Sorry about that, I got back two days ago, and I've been battling the flu or something since, so I've not caught up with my messages yet. I'll be getting caught up over the next week and will fix that access issue right now. Thanks.

This week there’s new $pages->saveFields() and $page->saveFields() methods on the core dev branch. You might already be familiar with the $pages->saveField($page, $field); method which lets you save one field from a page, or $page->save($field); which does the same. This is useful when you only need to save one field from a page, rather than the entire page. Now we have a plural version of that method, which lets you specify multiple fields on a page to save: $pages->saveFields($page, [ 'title', 'body', 'summary' ]); Below is the same thing, but on a $page object, so you don't need to specify a $page argument: $page->saveFields([ 'title', 'body', 'summary' ]); You can also use a string if you prefer: $page->saveFields('title,body,summary'); In my case, I needed this method for a project I'm working on, and I also needed it to save without updating the 'modified' time or user, which you can achieve by specifying the 'quiet' argument. Though note, the 'quiet' argument is available for all the page saving methods and has been around a long time. But I'm not sure how widely used it is, so I'll mention it. $page->saveFields('title,body,summary', [ 'quiet' => true ]); This week the API methods for Select Options fields have also been updated to add more convenience for getting, adding and removing options to an existing selection. Let's say we have an Options field of checkboxes named "colors". Each selectable option has an ID, optional value, and title. Now you can get, add, or remove by any of those properties. Previously you had to work directly with SelectableOption instances, which wasn't as convenient. // add by title of option 'Blue' $page->colors->addByTitle('Blue'); // remove the option with value 'orange' from colors field $page->colors->removeByValue('orange'); // get SelectableOption instance of option with title 'Red' $red = $page->colors->getByTitle('Red'); echo "ID, value, title: $red->id, $red->value, $red->title"; // check if colors has an option with value 'purple' if($page->colors->hasValue('purple')) { // it has purple } The methods added to SelectableOptionArray (not yet reflected in linked docs page) include: getByID($id) getByValue($value) getByTitle($title) addByID($id) addByValue($value) addByTitle($title) removeByID($id) removeByValue($value) removeByTitle($title) That's all for this week. There likely won't be any core updates next week, as I'll be out of town again. Thanks for reading and I hope that you all have a great weekend and great week ahead.

In the last couple of weeks I’ve been to several cities in Spain, France and Italy. I’d never been to any of those countries before (or Europe for that matter), so it was an exciting trip. Though the goal was for my kids to broaden their horizons and experience other parts of the world, as well as spend time with my parents and family. We got back this week and have been recovering from jet lag (another thing I’d not experienced before). The 6 hour difference was no problem getting there, but coming back, it’s a little harder to adjust! Next week I turn 50 years old (ugh), and then the following week I’m back in Europe again, except this time in the Netherlands on a bike trip with a client, and without my kids and parents. I’m not sure I’ll be able to do many core updates during the 10 day trip but do expect to have internet access this time, so will at least be online regularly and hope to be here in the forums. After that trip, I won’t be traveling again for a long time, and the focus will be on getting our next main/master version out. I noticed this week that @Robin S is now beating me as our most prolific module developer, with 72 modules! Great job and thanks for all the great modules Robin S.!

I mentioned a couple of weeks ago that I’ve got to do a lot of traveling in September and October, and so that means not a lot of core updates in the short term. Core activity may be a little quiet till the end of October. Sorry about that, I feel especially bad I’m not providing any good material for ProcessWire Weekly. But I’ll make up for it later, I promise! I’ve been reluctant to push any significant updates to the core because it is quite stable right now. So I’ll hold off on anything major till the traveling is done. I did just push one update to the dev branch that I think is a good addition though. I noticed recently that on a couple sites I work with, there was some markup cached with $cache that never seemed to expire. I tracked it down to an issue in WireCacheDatabase where some caches had ended up with invalid MySQL dates somehow or another. WireCacheDatabase now implements its own cache maintenance that ensures these never-expiring caches get deleted. In doing that, the cache maintenance process became more efficient as well, as it’s all handled with a single delete query, rather than multiple select and delete queries. If you’ve been noticing anything cached by $cache that doesn’t seem to be expiring when it should, it’s worth grabbing the current dev branch, or at least the updated /wire/core/WireCacheDatabase.php file from the current dev branch. One way you can tell if you have any caches that shouldn’t be sticking around are if you find any with dates prior to the year 1971, or any with a zero’d out date like 0000-00-00. Maybe there is still a bug to track down that is causing the occasional invalid dates, but at least now those caches won’t last more than 10 minutes. There likely won’t be an update here next week, as I’m anticipating no internet access, but there should be the week after next. Thanks for reading, have a great week and weekend!

@FireWire Apologies, I still don't completely follow what you are trying to do in the code above, but wanted to comment about a couple of things. This is because at this point in your code, you've only dealt with the Field object (or in this case a CustomField object), and no $page has been involved. Since values are stored with pages, all you've got here is a set of blank Inputfields, which probably isn't useful for anything. In this case you are iterating that Field object, which I don't think has any value. What you want to iterate is the value from the page. So if your CustomField is named "custom_field": foreach($page->custom_field as $property => $value) { echo "<li>$property: $value</li>"; } Are you setting an 'addClass' property to your Inputfield definitions in your /site/templates/custom-fields/field_name.php file? And you want to use the value of that property somehow on the front-end of your site? That property is for adding a class to the Inputfield in the admin, but if you want to have access to it on the front-end of your site, I suppose you could do this: $defs = $fields->get('custom_field')->defs(); /** @var CustomFieldDefs $defs */ foreach($page->custom_field as $property => $value) { $f = $defs->getPropertyInputfield($property); echo "<li>addClass for $property is: $f->addClass</li>"; } But you might also just consider going straight to the source, by including your field definitions php file directly: $defs = include('./custom-fields/field_name.php'); /** @var array $defs */ foreach($page->custom_field as $property => $value) { $def = $defs[$property]; if(isset($def['addClass'])) { echo "<li>addClass for $property is: $def[addClass]</li>"; } } Note this will only work if you don't have your properties nested within fieldsets. If they are nested in fieldsets, you can still do it, but you'd just need to account for that in the code. You wouldn't need to account for it in the example above this one.

Looks interesting! I just created an account https://pinkary.com/@processwire. I'm not sure what happened to Twitter but seems like it's gone downhill. I don't have an appetite for it. I've only kept the ProcessWire account on Twitter to post links to new blog posts, but not sure I'll keep doing that. Threads seems a lot better, but I don't think there's much of a webdev community there, that I've found anyway. This forum is my favorite social network. I look forward to trying out Pinkary more.

@FireWire You can just iterate $page->your_custom_field as if it were an array. There are a couple of foreach() examples in the blog post, in the section headlined "Outputting custom fields". Though let me know if I've misunderstood what you are looking for.

As far as JSON column types, the module lets you choose between JSON, TEXT, MEDIUMTEXT and LONGTEXT (or is it BIGTEXT, I can't remember). The main difference between them is how many kilobytes/megabytes/gigabytes you can hold. Functionally I can't tell any difference between them, and you can easily switch between them in the module settings. But as I understand it, JSON columns have the benefit of being more optimized for MySQL JSON-based queries, even if those queries still work on the text column types. I expect there may be a measurable difference at larger scale that isn't yet apparent at the scale I'm currently working at. The downside with the JSON column type is that you can't have a FULLTEXT index. So you can query individual subfields/properties, but can't perform a text search on all of them at once. As I understand it, JSON column types also have the benefit of being able to support MySQL 8 multivalue indexes. These enable you to pick and choose which individual fields within the JSON you want to index separately, or combine several of them in one index. I plan to support these with CustomFields in a future version. For now, I find MEDIUMTEXT to be a good fit for my project, as I do like to be able to perform text searches on the entire field at once, while also being able to query individual fields within it.

I've got a lot of travel coming up in the weeks ahead, so there may be a few quieter-than-usual weeks in terms of core updates. I'll be in and out of town a few times, and I'm not much of a traveler, so will see how it goes. I'm not yet sure whether I can do work remotely, or what will be available in terms of internet access. There's a lot of client work to wrap up before hitting the road, so I've been focused on that this week and will have to next week as well. It's all ProcessWire related work though, so still having fun. There have been a few core updates this week, and there will likely continue to be in the coming weeks, but just not major core updates. By November hopefully all will be back to normal in terms of schedule. I'll be focused on getting a new main/master version out then. I also expect to have a new version of the CustomFields module (from last week's blog post) ready in the next week or so as well. Thanks for reading and have a great weekend!

@Robin S These property/subfield definitions are in files rather than the database, so there are no database-style IDs. Or, you can think of the property names as the IDs. There is already is a to-do note in the module to add support for property aliases, so that you can rename properties without having to convert data. That's not in this v1 beta version, but likely will be in the next one. That will enable you to rename properties when/if the need arises. But you'll still have to update your own code that refers to any of those names, as would be the case with any other field. When it comes to deleting properties, the no-longer-needed data would be cleaned up whenever it is saved. This is like any other Fieldtype that encodes multiple properties/subfields together (Textareas is one example, Combo is another, depending on the chosen storage method). If you regularly need to rename and delete these kinds of things after development of a site, regular old ProcessWire fields (without subfields) are hard to beat. But either way, you still have to consider your own code that's referring to those fields. Thanks, I will correct the typo!

@Jim Bailie If I understand the question correctly, you want to convert several regular ProcessWire fields into a single Custom Field? There isn't an automated way to do that. I suppose there could be though, as they are using all the same configuration properties.

@Jim Bailie They are defined just in a PHP (or JSON) file, so exporting (or importing) the definitions would be just a matter of copying the file from one system to another.

This week we introduce a new module named Custom Fields. This module provides a way to rapidly build out ProcessWire fields that contain any number of subfields/properties within them. No matter how simple or complex your needs are, Custom Fields makes your job faster and easier. Not only does this post introduce Custom Fields, but also documents how to use them and includes numerous examples— https://processwire.com/blog/posts/custom-fields-module/

Strange I'm using Chrome on Mac too, is anyone else able to duplicate that issue?

@adrian I'm not seeing that here, maybe it's browser specific - what browser are you seeing that issue in?

This week the most useful core update is likely the refactored column width slider in the template editor, located Setup > Templates > [your template] > Basics > Fields. You may or may not already know that clicking, holding and dragging the percent indicator on the right side of each field adjusts the column width. With the term “column width”, I mean the width of the field in the page editor, for when you want to have multiple fields in different columns on the same row. It’s a convenient and time saving shortcut. But it was also a little tricky to use, as it allowed anything between 10% and 100% in 1% increments, and it was a little finicky trying to get the percentages just right sometimes. It’s something that’s been bugging me for awhile, and @Pete messaged me on Slack this week and mentioned it. He suggested making it operate in 5% increments rather than 1% increments. He also suggested making a double click of the percent indicator open up the dedicated column width range slider that allows for more precise adjustments. I thought those were good suggestions, so I went ahead and implemented them this week. In addition to now using 5% increments, it also supports the commonly used 33%, 34% and 66% width values as well. But if you happen to already have some field that is using a less common width, like 27% or 72%, etc., then it reverts back to 1% increments for the same behavior as before. Of course, you can also use the 1% increments by double clicking the percent indicator to open the dedicated column width range slider. Thanks Pete for the suggestions, I think it all works better now. I’ll be applying the same changes to FormBuilder’s equivalent of this feature as well. This week I’ve also been working on the new CustomFields modules (FieldtypeCustom and InputfieldCustom). Most recently I’ve been working on adding support for multi-language fields, as well as adding more examples and tools to make it really easy to use and configure. I may have it ready as soon as next week or the following week. The PageAutosave module is also getting a new version soon. I’ve been focused on the LivePreview feature of it and making a version of it that doesn’t depend on auto-save. The alternative LivePreview option (which we’ll just call “Preview”) will work anywhere because it has no field limitations. It simply updates the preview window whenever you save the page. While that’s not as fancy as live preview as-you-type, it’s still very helpful, while being reliable in any situation. It’s reliable and portable enough that I may end up putting the feature in the core, but will be testing it out in the next version of the PageAutosave module first. Have a great weekend!

We couldn't use a cloud version for the core of course. TinyMCE 7.x version would be bundled in InputfieldTinyMCE the same way the current one is. It may be that the cloud version is used for the Pro module though (if that is built), as I think that's how the company distributes their commercial version, but I don't really know. I'll be sure to ask more about it when the time comes. Interestingly, the person I talked to seemed to be involved with both CKEditor and TinyMCE. It appears they are now owned by the same company: https://www.tiugotech.com/tools/

I was able to speak with someone at TinyMCE on a Zoom call this week and we had a good meeting. They are going to make it possible for us to continue using TinyMCE 7.x+ in the core, even though it is using a GPL license, while we use the MPL 2.x license. They will make a custom license available for ProcessWire and I hope to have the details of that potentially next week. We’ll have to review the conditions and everything to make all is good, but it sounds like it will very likely solve the issue for us. I’m really happy about this and look forward to working with TinyMCE 7.x They also expressed interest in us potentially collaborating on a separate module that would make some of the commercial and advanced TinyMCE features available to ProcessWire users that wanted them via a paid service, like a Pro module. This option hasn't been available to us before, so I thought it sounded interesting. I'll definitely be communicating with them more about that to see what's possible. That’s all I know so far, but will keep you up-to-date as I learn more.

This week I've bumped the dev branch version to 3.0.241. Relative to the previous version, this one has 29 commits with a mixture of issue resolutions, new features and improvements, and other minor updates. A couple of PRs were also added today as well. This week I've also continued work on the FieldtypeCustom module that I mentioned last week. There were some questions about its storage model and whether you could query its properties from $pages->find() selectors (the answer is yes). Since the properties in a custom field are not fixed, and can change according to your own code and runtime logic, it doesn't map to a traditional DB table-and-column structure. That's not ideal when it comes to query-ability. But thankfully MySQL (5.7.8 and newer) supports a JSON column type and has the ability to match properties in JSON columns in a manner similar to how it can match them in traditional DB columns. Though the actual MySQL syntax to do it is a little cryptic, but thankfully we have ProcessWire selectors to make it simple. (It works the same as querying any other kind of field with subfields). MySQL can also support this with JSON encoded in a more traditional TEXT column with some reduced efficiency, though with the added benefit of supporting a FULLTEXT index. (Whereas the JSON column type does not support that type of index). For this reason, FieldtypeCustom supports both JSON and TEXT/MEDIUMTEXT/LONGTEXT column types. So you can choose whether you want to maximize the efficiency of column-level queries, or add the ability to perform text matching on all columns at once with a fulltext index. While I'm certain it's not as efficient as having separate columns in a table, I have been successfully using the same solution in the last few versions of FormBuilder (entries), and have found it works quite well. More soon. Thanks for reading and have a great weekend!

Since it looks like there is a some crossover with Mystique, and it also looks like that module is active and supported, I'll release this module in ProFields instead. That way it's not competing with Mystique, which looks to already be a great module. I'll focus on making Custom fields have some features that maybe aren't available in Mystique, like ability to query from pages.find and perhaps supporting some more types, etc. Plus, the new module fits right in with the purpose of ProFields and a good alternative to Combo where each have a little bit different benefits to solve similar needs.

@Jonathan Lahijani Sort of, but it's not just that. I'll go into the details once I've got the storage part fully built out. It supports any Inputfield type that can be used independently of a Fieldtype. Meaning, the same types that could be used in a module configuration or a FormBuilder form, etc. It can be used IN repeater fields (tested and working), and likely in file/image custom fields as well (though not yet tested). But you wouldn't be able to use repeaters or files/images as subfields within the custom field. Files/images will likely be possible in a later version though, as what's possible in Combo will also be possible here. Yes, if you use the .php definition option (rather than the JSON definition option) then there are $page and $field variables in scope to your .php definition file. @MrSnoozles Searching yes. Can be used in repeaters (yes), but can't have repeaters used as subfield/column types within it. @poljpocket Yes. While I've not fully developed the selectors part of it yet, the plan is that it will support selectors with subfields for searching from $pages->find(), etc. @wbmnfktr The API side of this should work like most other Fieldtypes with subfields, so I think it should be relatively simple for importing and may not need any special considerations there, but I've not tried using it with a module like ImportPagesCSV yet.