Kiwi Chris

A question: Do new and changed fields and/or pages get processed in the order they're specified? I'm thinking of the scenario where a new page field is added that depends on a template and page that may also need to be added. That's one of the scenarios where using the built in field inport/export doesn't work too well if there are dependencies. It will advise you of them, but then you have to go back and import the dependencies then re-do the import, whereas if it's possible to specify order so that dependencies are met before an object is processed, it would make migrations more robust. This isn't an issue for declarative development via the API rather than the admin UI, as the developer can order their code so that dependencies are processed in order, but if changes are made via the admin UI this is a potential problem.

I figured out the problem. The hosting control panel said that PHP 7.4 was installed but it was actually only supplying 5.4. I got that fixed and the module works now.

That works perfectly. It saved me a heap of time trying to figure out what hook to use.

I need to restrict editing access based on both the user role, and the value of a sub-field in a page reference field. eg $page->competition->closingDate There's some discussion about restricting editing access based on role, and that works, but when I try to access the page via the method @Robin S suggests, the page returned is an admin page rather than the page being edited, so I don't have access to the edited page's fields.

@ryan, this module needs to be updated to support import of decimal fields now they're in the core.

I get the following error on attempting to import a CSV file: I'm running ProcessWire 3.0.171, on PHP 7.4x

I've looked at quite a few other CMS/CMFs and most of them fall down in some respect compared to ProcessWire, whether it be documentation, community, or ease of use. That doesn't make ProcessWire perfect, and given the diversity of usage scenarios there will always be things that could be done differently. The admin certainly can be a mixed blessing. It's so easy to use, that it's tempting to rely on it at least to start with, but then when you need to maintain changes, it can get complicated. I think what could be quite nice, if it's possible, would be to have the ability to enable change tracking for templates and fields, when they're modified via the admin. It's already possible to export and import field and template definitions as JSON, so combine this with the ability to track what's changed, and something like @bernhard's migrations module, and it could be easier to combine the ease of using ProcessWire admin to make changes, and the need to be able to roll out changes from development to production sites.

@bernhard Thanks for a really helpful example. It's also good to know other people are using Processwire as a full backend application. Thanks also for making RockFinder3, it's a great module. For non-editable tabular lists, I've just used the core MarkupAdminDataTable that works well enough for my needs, although I should probably build some sort of reporting module so that I can easily define group headers and footers for totals, headings etc.

The "Unique" status in Processwire does quite the opposite of what I want. What I need is to be able to have First name: John, last name: Smith, Address: 10 main street allowed but unique and also First name: John, last name: Smith, Address: 10 high street also allowed but unique Address can't be the unique field though because there could be someone else living at the same address, so what makes a record unique is the combination of all three fields. This is easy to do in SQL, eg CREATE UNIQUE INDEX person ON members (firstName, lastName, address); but not quite so obvious in Processwire, although I think perhaps a hook before Pages::added could do a $pages->count('firstName=John,lastName=Smith,Address=10 main street') and if this returns a non-zero value, return an error. I'm not sure what sort of performance implications there would be doing it as a hook vs in SQL on multiple fields in a table.

I can write a hook to make the page name the page ID. It's only one duplicate field, and I can rename the display name for title for the templates involved as First Name I guess, as that's always going to be a required field, just not unique. It doesn't solve the issue around validation of uniqueness across multiple fields, although I guess that can be achieved with a hook before saving, to check whether $pages->count() > 0 for the field combination I want to be unique.

From recent discussions around the future of Processwire, I suspect my use case might be a bit different to others, as rather than using it in the website 'builder' type scenario, I'm using it more as a database management system, where structured data is critical, and quite often 'pages' will never be visualised via a template, and if they are, the template file is effectively a 'report'. In my case I'm tending to build sites where the admin IS the site, and there's often little or no front-end. I've wondered whether I'd be better to learn some framework like Yii or ASP.Net that have CRUD code generators that can generate data entry forms for SQL tables, but I like Processwire's admin, and the permissions system is something I rely on a lot. I've started working on a membership app, that will have quite a few users, and I've immediately run into a problem. I want admin users to be able to store first name, last name, address etc for members, but it's quite conceivable with a large enough list that it's possible to have more than one say 'John Smith', living at different addresses. I know how to write a hook to put an arbitrary auto-incrementing number in the title field (which will also populate the name field), but this seems to me to break the DRY principle. There's already a page ID, and if I'm effectively populating two other indexed database fields with an arbitrary number so that I can uniquely identify records and keep Processwire happy, it doesn't feel quite right. In straight SQL I can use an auto-increment numeric surrogate primary key and create a constraint on multiple columns so I could have as many 'John Smiths' as I like as long as they reside at different addresses, but I'm not sure how I'd go about this in Processwire, as each field resides in its own table. Perhaps the new Combo Profield @ryan recently released might be have potential for this, as all the subfields reside in a common database table, and there's already some capability to edit the database schema, although I'm not sure if this extends to creating custom constraints? That would be a really great feature, as it would allow defining unique records based on a combination of sub-fields. The Combo fieldtype is still a fieldtype though which needs to be added to a template with mandatory fields, so I still potentially have page id, name, and title all essentially duplicating each other's functionality as arbitrary numeric fields. I understand why page name is meant to be mandatory, as a page that has a template file needs to be accessible via a URL, and the page name is part of the routing, however I'm not sure whether it's practical for pages that don't have a template file to simply work off the page ID? What might be useful is a template property setting that can indicate that the id should be used as the name as well, so there's still a 'name' but it's populated at runtime as a reference to the page id. I'm guessing it might be possible to do something like this via a hook?

Nice clean site. Although I'm on the other side of the world, I remember seeing Shetland calendars and other things most years, because in the pre-internet days Mum made a pen-friend connection with a Shetland woman with the same first name as her. I believe the two were put in contact by a publication run by Enid Blyton - yes The Enid Blyton of Famous Five fame. Mum's never been to Shetland nor has her pen friend ever been to New Zealand, but in our family we've always had this kind of long distance connection. I have an idea that my brother's mother-in-law actually has relatives from Shetland, so the connection there is even stronger.

Most likely I wouldn't be able to make any more commits to the core if I got run over by a train. I've always thought this was one of the reasons to use open source, ensuring that when someone smacks the tracks, the code doesn't. But it's not just about being open source, the code also has to be clean and well documented so that others can easily take it on board. That's one reason why I put so much effort into code quality and code documentation. My intention is that the code is always ready for others to understand and work with. As for commercial services/modules, it's the same risk inherent with any product or service you pay for. While not open source, the Pro modules do have the same code quality and documentation as the core, and are not obfuscated or encrypted. If it sets your mind at ease, I'm 46 years old (not 76), and am very healthy. I run and lift every day, eat lots of salads, don't eat red meat, and don't participate in any dangerous activities. Most likely I'll be here for at least another 46 years. But if I'm ever derailed then I know the project would still be in good hands. Never say never. When I was 25, I was fit an healthy, and then got meningitis which nearly killed me. When I was 42, I was also apparently fit and healthy, and then ended up with a twisted bowel which also nearly killed me. I've recently been asked a similar question by some of my clients, "What happens if something happens to you?" I think perhaps a peace of mind option with the pro modules might be to include some sort of clause that allows rights to modify them to provide bug fixes etc, and also explicitly have some sort of clause around what happens if the developer (you) ceases to support them. I've often thought about software licensing and what should happen if the original author is no longer able to support their code, but there's still a need to use it. I know that people need to earn a living, and relying on donations usually won't be sufficient, so licensing software is a fair way to get paid, however it would be useful to have some sort of commercial license that both respects a copyright holders right to earn an income, but also provides peace of mind to licensees in the event of the software being abandoned by the original creator. I understand in normal circumstances, copyright extends past an author's death, but I know that my wife or daughter would not be able to support any code I might write, although they might need revenues generated by it if it's still useful without modification, so the licensing thing is complex.

Thanks. I worked out the first approach which works as a quick fix, but it's good to know the second one will work as well, as I think from an OO programming perspective, it's clearer to use the second option as I am effectively setting a page property, just I'm doing it at runtime, and as a non-persisted property.

Every week or so, I'm finding large amounts of space taken up with orphaned image files on some pages that have an image field but no images, with the image file names starting with a base number then numbered sequentially. I the worst cases this can amount to several hundred images. I run code similar to what @ryan posted in this thread: however after a few days the large number of files are all back again taking up considerable amounts of space. These are pages that no one other than myself has editing access for. The one thing the pages in question have in common is that they do have child pages that have images, and the orphaned files that are duplicated many times are an image that belongs to the first of the sub-pages of the page where the large number of copies are created. I have this function: /* * If a page doesn't have a header image explicitly set, search for the first child page that has a header image, and use that instead. */ function getHeaderImage($page) { if ($page->headerImage) { $headerImage = $page->headerImage; } else { foreach ($page->children as $item) { if ($item->headerImage) { $headerImage = $item->headerImage; break; } } } return $headerImage; } And in the templates affected this: $page->headerImage = getHeaderImage($page); It appears that every time I assign an image this way, a new copy is getting made into the page's /site/assets/files/ folder even though I never save $page. Normally with other page properties, if they're set, but never saved, then they don't persist, but I know images may be different as they correspond to actual files on disk. Is this expected behaviour?

I agree. The way I see it, if there's a low level SQL type that supports given functionality, there should be a fieldtype that corresponds to it, however the ProcessWire specific fieldtypes are there for when additional functionality is required that can't be supported directly through SQL. Speaking of that, it may be a bit of a limited use case, but I've wondered a few times about adding geometry fieldtypes and adding geometry selectors. There's already the FieldtypeMapMarker, but this is using just standard float fields in mySQL, while mySQL and MariaDB have support for geometry datatypes and spacial indexes, and there are times it would be nice to be able to do a query and return all the places within a given distance. I already hacked the FieldtypeMapMarker a bit to allow adding KML overlays exported from Google Earth, or mobile tracker apps, but there's certainly a lot more that could be done to give geospatial data first class treatment if enough people use it.

Tabulator looks interesting, although sometimes I want forms, not tables for data input. The trouble with Profields Table is it seems to be geared around having multiple records in a row format per page. If I understand it correctly, the new combo fieldtype stores things in sql fields in a table in the database, but gives more flexibility around layout, and is a single record per page, so this is much closer to what I want, and may be what tips the balance in terms of me buying the ProFields package. A question though is whether this is actually creating separate fields in the database table, or doing what custom image fields does, and just storing a JSON object in a single field. From an SQL perspective, having separate fields and being able to do sorting and grouping at the SQL level would be more convenient than doing it at the PHP level. I would like the option that I raised elsewhere around ordinary field types, of being able to choose whether to index subfields or not, as some will only ever be retrieved based on selecting on other fields. I guess I should bite the bullet and try building my own custom fieldtypes. I'm at the point I'm comfortable building simple Process admin modules thanks to @bernhard and his excellent tutorial on building custom admin modules, however I think a good tutorial on building custom data driven fieldtypes or custom admin data access modules might be helpful, and it might actually be as useful as adding more features to ProcessWire itself, as with all the various inputfields and Process modules, possibly everything I need is already there for custom SQL data, just I need to join the dots.

I'd say this is a biggie. During lockdown here in NZ (thankfully long over), I had the need to move various client databases to the cloud, and started looking at a headless CMS, Directus, because ProcessWire doesn't let you easily hook up an SQL database, leaving the SQL structures alone and just give you an admin UI and API, and yet looking at what Directus does and what ProcessWire does, I can't see why ProcessWire couldn't be made simple to connect to an existing SQL backend. Input Fields work with a wide range of data, and there are already ProFields like Table and Combo that connect to an SQL table behind the scenes, so maybe having a fieldtype (maybe a pro field?) that can connect to any SQL table would be quite handy. That would make connecting to existing apps that have been built outside of ProcessWire straightforward, and also make migrating data simple if it does make sense to pull it into the ProcessWire way of doing fields, as the external data can be accessed via the API. ProcessWire already has a fields table that contains the definitions for how each field should be managed in the backend, and I guess it should be possible to do something similar linking an existing SQL table, where it is possible to specify any inputfield parameters for each table field, while leaving the actual SQL table alone. I can see a down side, in that some people who are used to SQL tables, and don't quite understand the ProcessWire way might be inclined to use SQL tables for everything when the ProcessWire way of one table per field actually has advantages depending on the data, but if it makes migration easier, and sticks with ProcessWire's philosophy of trying to keep out of your way and not dictate how you work with your data, then perhaps it could be useful?

For years, as a personal project, I've been documenting my little corner of the globe via my personal website www.marlboroughonline.co.nz, which also gets used as my testbed for new ideas with ProcessWire, SEO and other stuff, as I'm not going to have some client on the phone if I break something, but if I do it well, I can earn a few dollars from Google adsense. I've always had a bit of a passion for natural history, being introduced to David Attenborough documentaries when I was a teenager, and actually studying life sciences at university, so it's only natural if you'll excuse the pun, that a part of my site would be dedicated to documenting local plant and animal species in my part of the world. For many years, one of go to reference websites for information about native plants has been New Zealand Plant Conservation Network, and I often include a link from my site to details on a species on the site as an authoritative reference, and I use the site to help identify plants I've photographed. It was only recently that for some reason I decided to have a look at the source code of the site, and it looked suspiciously like it was made with ProcessWire. After a bit more detective work, I found out that it was made by @Robin S and is indeed built with ProcessWire. To get away from the computer, I also teach kids edible gardens at my daughter's school a couple of hours a week. It's part of a wider Enviroschools programme here in NZ, and among other things it covers is teaching kids to recognise noxious weeds. There's a really useful website Weedbusters that's supported by councils and used as a reference for many invasive plant species, and it turns out it's more of Robin's handiwork. I think it's a bit funny that even when I'm supposedly away from ProcessWire engaging in other activities, it seems to follow me around. ?

There's also the wishlist and roadmap on the forums here. https://processwire.com/talk/forum/5-wishlist-roadmap/ @ryan from your perspective do you have a preference where people post feature requests, and is there a reason to use the forum vs github and vice versa? Interesting feature request. I could have used that recently, although in most cases the images I wanted to use were the only ones on the page they belonged to, so I just used a normal page reference and used the image from the page. For pages with more than one image though, it wouldn't work if I wanted to pick a single image. Now that images support additional fields via a template, then I assume behind the scenes there is a page holding this data, so it should be possible use selectors like with regular pages on these fields to find images that match regardless of what page they belong to? That could apply not only for an images reference field, but also for the RTE when inserting images, to provide an image search functionality, like currently is possible when inserting links.

Since others are getting into localsied greetings, Meri Kirihimete! (Although we're actually already a day ahead and Christmas was yesterday in my part of the world.)

Currently, FieldtypeTextarea and FieldtypeText automatically have a fulltext index created on the field content, however there may be scenarios where this has an undesirable impact on performance and storage if the content of the field will never be searched, ie a field will only ever be accessed as a result of selectors using other fields on templates where it occurs. An example where I experienced an issue with the default behaviour was a module that stored a log of data import to a FieldtypeTextarea field, which grew quite long when importing thousands of records. The fulltext index of course indexed the content of this field, although it was unnecessary, resulting in a very large index file. What would be useful, would be an option (maybe a checkbox) when creating a field with one of these fieldtypes to disable indexing. This would mean the default behaviour would remain with the field indexed as usual, however if the field is never going to be used in selectors, checking the box would drop any fulltext or ordinary index, providing performance benefits. Obviously there needs to be a warning associated with checking this option that the field will no longer be able to be found using selectors, and unchecking the option subsequently should create the index(es) if required.

I recently did an import of about 15K records with these modules, and they worked well, however there was one undesirable side effect of storing the log to a standard Processwire text_area field, log_messages. Processwire creates a full text index on text_area and text field types. I had a few issues with field mappings and defining unique identifiers the first couple of times I ran the import, so just used Tracy Debugger to delete the pages, fix the import definition, and run the import again. Unfortunately, over my efforts, the full text index on the field log_messages grew to over 2GB in size, even though the imported CSV file itself was only about 1MB. It took me a bit to figure out why the database had suddenly grown from a few MB to several GB, and once I located the problem, I ran Optimise Table in mySQLAdmin, which cleaned things up and got things back down to a normal size. I suspect in this case, the log_messages field probably doesn't need to have a full text index on it, although I guess if someone has a large number of import pages, maybe there is a case for them to be searchable? If there isn't a need for the log_messages field to be searchable, then perhaps a custom text fieldtype without a full text index might make both the import run more quickly if the field is repeatedly updated during import, and also avoid the issue of creating an enormous index file. I've made a post to the Wishlist and roadmap thread, suggesting an option for textarea fields to have indexing disabled if not required, as this would resolve the issue.

I have a site with about 16K pages, with most of these (15K + ) being templates with just two text fields (including page title), along with several page reference fields, and a couple of numeric fields and all the tables take up just over 20MB of storage, however there is a huge full text index file of over 2.1GB that seems to be totally out of proportion to the size of the database. Does anyone know whether this is normal behaviour with Processwire? From the reading I've done, it seems a full text index of similar size to the table it indexes is not unexpected, but 100x the size of all the tables in the database seems excessive. If this isn't normal, what's the best way to fix this? Edit: It turns out the huge index was related to logging of imports to a text field by this module: I'll post in that thread regarding the issue since it's module related.

@netcarver Here's details direct from PHP manual: https://www.php.net/manual/en/intro.pcntl.php It certainly doesn't recommend PCNTL be enabled for a web server environment. In the user contributed notes, there is a comment about undefined functions which matches what I was experiencing.