Leaderboard

This week I didn't get as much time as I'd hoped to work on the core, but still managed to get a few things done. I did get some small core updates in place, but not enough for a version bump, and got the second covid vaccine, kids back to in-person school (3 hours a day, after a year of being away), and a couple more major module upgrades released. In last week's post, someone asked for an update to the ProcessWireUpgrade module, and I had a look and a realized it was long overdue for one, so I tried to cover a lot of ground there. The new version is now out and improved throughout. Grab this latest version of ProcessWireUpgrade if you are interested. It's always been pretty useful, but it's even more so now. I've also updated the modules directory feed so that it can let you know when Pro module upgrades are available as well (as requested). The ProcessWireUpgrade module reports this Pro module information in a clear manner too. Some improvements have also been made to the core version updating features as well. The other module updated was FieldtypeEvents, a module that demonstrates how to create simple a Fieldtype+Inputfield module pair that has repeatable rows of multiple columns. The last version was made for PW 2.x, so I largely rewrote the module while updating it for PW 3.x. It's been simplified quite a bit so that it can be even more clear how to adapt it to create your own custom module. Being PW 3.x exclusive also opened more doors in keeping it simple but powerful. If you don't mind doing a little customization in adapting the code for your purposes, this module can be every bit as powerful in solving custom needs as Repeaters, but without Repeater overhead. I updated this module because another ProcessWire user was asking about how to store multiple columns of repeatable data for 20-million pages. Repeaters would be too much overhead at that scale, though FieldtypeTable could fit the bill. But at such a large scale, I thought a custom Fieldtype would be even better. So FieldtypeEvents shows you how to do it, and it's [hopefully] simpler than you might think. This has nothing to do with PW, but to follow up from last week, that second dose of the covid vaccine was an interesting experience. I got the shot (jab?) and strangely didn't feel a thing. I walked home and took the band-aid off my arm and noticed there was no sign of anything, no little pin prick or red spot, and a completely clean band-aid. I started to wonder if I'd been fooled or something. If someone sticks a needle in your arm, you should feel it right? And there should be some evidence of it? Well there wasn't (and no I didn't watch the needle, who would do that?). They also told me I'd feel ill for a day or two, and I felt completely fine even the next morning. But then 10 am rolled around and my arm started feeling sore and I was suddenly very tired. So tired that it was laborious to walk, sit at the computer, or do anything other than chill on the couch. That might sound undesirable, but it wasn't uncomfortable, and it was kind of funny being so ridiculously tired, but also really relieved. It meant it was working and my body was getting a download on how to battle covid, and putting all its resources into taking it seriously. It was fascinating, so I just crashed on the couch, relaxed and let it do its thing. By the next day, full energy was back, batteries recharged, upgrades installed. From what I understand, some people have no side effects, but I was one of those that did. Though if anything I enjoyed the different experience for a day, and the comfort of clear evidence it was working.

modules / core / process page list

@fruid I agree with you that at first it looks complicated, mainly because it need a compilation step which has to be handled in some way. Nowadays is quite common to rely on tools like webpack, rollup, parcel, vite, etc.. to handle some activities like compilation, minification, bundling, etc. Postcss is mainly just one of them, but specifically intended to work with with css. Once you grasp the few key steps to install it and how to purge your css efficiently I guarantee you will see the benefits at your disposal (mainly speed of layout blocking and very low css file sizes). If someone it's interested I can make a video where I can explain all the install steps in further details.

@benbyf - I am not running any sites on it yet and I actually don't think I will until I know that Ryan has 8 installed on his dev machine. He still seems to be in "new feature mode" rather than "bug fix mode" at the moment so I can see some of these PHP 8 bugs lingering for a while. I am still running 8 on my local dev machine but these days this is mostly for module development and not much else, so I am not coming across these other bugs yet.

This sounds like an SNI problem, although Windows 7 should support SNI. https://www.kinamo.be/en/support/faq/which-browsers-support-server-name-indication-sni The other possibility could be root certificates that aren't recognised by Windows 7, or TLS/SSL versions supported by the server. A lot of servers have retired older versions of TLS because they're insecure, and by default Windows 7 doesn't support newer versions. https://www.zdnet.com/article/nsa-urges-system-administrators-to-replace-obsolete-tls-protocols/ If this is the case, changing the CMS won't resolve the problem, as the problem is an issue between the server and Windows 7 not the CMS. This might help you: https://help.runbox.com/enabling-tls-1-2-on-windows-7/ If enabling TLS 1.2 on Windows 7 makes the website work with Windows 7, then you have the answer to what is causing the problem. Unfortunately you can't be sure that anyone running Windows 7 has enabled TLS 1.2, and if the server has disabled older versions for security, it's unlikely to be able to get insecure versions of TLS enabled again, as it puts the server at risk. It's over a year now since Windows 7 reached end of life, so anyone still using it on the internet is taking big risks with their security.

@Hector Nguyen getcsv() does return an array, but it is an array representing and consuming the memory of just one line from a CSV file (i.e. the columns from 1 row). fgetcsv() is just a layer on top of the fgets() function, which reads one line at a time from a file, which is what makes it memory friendly. On the other hand, PHP functions like file() or file_get_contents() do read the entire file in memory, so they are not memory friendly, even if they are fast. @fedeb I think the best route to take for your groupID+start+end+sequence would be a custom Fieldtype. This would give you all of the benefits of having a repeater, and without any of the overhead. Custom fieldtype may sound complicated, but it's not at all. I've developed a module that can very easily be adapted for this need. See FieldtypeEvents which was created as an example to build exactly this sort of thing from. If you are interested in that route and have any questions, I'm happy to walk you through it.

In InputfieldURL I added a website link without the protocol at the beginning. It showed me this message: Maybe it is time to change it to https:// instead.

101aandd.com New site mostly designed by the client and implemented in Processwire. Mostly concentrating on small subtle animations.

Nice! Thank you for the help!

@3fingers why is this so complicated? I thought an npm would do most of this work for me. I looked at some tailwind css tutorial (unrelated to pw), looked quite simple but I didn't know which folder to install tailwind into in my pw installation using terminal, so I came here. All of a sudden: install this, download that, run this code, then this, purge css, post css - super complicated! Why can't it be as simple as uikit?

Hi @OrinTerry, I think it is the problem of Windows 7 rather than the CMS system. I am afraid that the problem may still persist after you migrate to ProcessWire. Gideon

Finally I got my local setup switched today. PHP 8 which needed a new apache version too, and then SSL needs to be updated, etc., etc., ... Now I need to upgrade my IDE for PHP 8 support too. ? After all I could debug and correct the WireMail-SMTP module: https://github.com/horst-n/WireMailSmtp

@AndZyk Alright, I had some fun with it. Here's an improved script for the asset export, which can handle nested repeater and matrix repeater fields: /** * Get a flat array of all images on the given page, even in nested repeater / repeater matrix fields. * * @var Page $page The page to get the images from. * @return Pageimage[] Array of Pageimage objects. */ function getImages(Page $page): WireArray { $images = new WireArray(); $fields = $page->fields->each('name'); foreach ($fields as $field) { $value = $page->get($field); $type = $page->fields->{$field}->getFieldType(); if ($value instanceof Pageimage) { $images->add($value); } elseif ($value instanceof Pageimages) { $images->import($value->getArray()); } elseif ($value instanceof RepeaterMatrixPageArray || $value instanceof RepeaterPageArray) { foreach ($value as $repeaterPage) { $images->import(getImages($repeaterPage)); } } elseif ($value instanceof RepeaterMatrixPage || $value instanceof RepeaterPage) { $images->import(getImages($value)); } } return $images; } $images = getImages($page); // create target folder for the page assets $targetDir = $config->paths->assets . 'export/' . $page->name . '/'; $files->mkdir($targetDir, true); // move all images to the target folder foreach ($images as $image) { $name = $image->basename(); $target = $targetDir . $name; $src = $image->filename(); $files->copy($src, $target); } This could be extended in any number of ways: Handle file fields as well as images. Handle any other fields and dump them in the target folder as JSON. Handle native Page properties. At some point you got a complete page export module. Might want to look into the pages export module if you don't want to write all that stuff yourself ?

Hi, Thanks a lot for all the feedback. I did some additional tests based on all of the suggestions you gave me and results are already amazing!! Figure 1 shows @ryan suggestions tested independently: 1. I created the $template variable outside the loop. 2. I created the $parent variable outside the loop. The boost in performance is surprising! Defining the $parent outside the loop made a huge difference (before I didn't assigned the parent explicitly, it was already defined in the template thus the assignment was automatic) 4. I also tried this suggestion ($page->name = "protein" . $i;) and although it seems to boost a bit performance I didn't include the plot because results were not conclusive. Still I will include this in my code. Figure 2 is based on @horst suggestion. I tested the impact of calling gc_collect_cycles() and $pages->uncacheAll() after every $database->commit(). I didn't do a test for $pages->uncache($page) because I thought $pages->uncacheAll() was basically the same. Maybe this is not true (?). Results don't show any well defined boost in performance (I guess ryan's recent reply predicted this). I still need to try @BitPoet suggestion because I am sure this is something that will boost performance. I am now doing this tests on my personal computer. I will do this test when running on the dedicated server. I will also would like to try generators (first time a hear about them ) ______________________________________________________________________________________________________________________________________________________________________________________________ One last thing regarding the fields in the protein template and the data structure in general (the pseudo code I posted initially was just as an example). Proteins are classified into groups. Each protein can belong to more than one groups (max. 5). My original idea was to use repeaters because for each protein I have the following information repeated: GroupID [integer], start [integer], end [integer], sequence [text] The idea is that from GroupID you can go to the particular group page (I have around 50k groups) but I don't necessarily need a page reference for this. The csv is structured as follow. Note that some protein entries are repeated which means that I shouldn't create a new page but add an entry to the repeater field. Protein-name groupID start end sequence A0A151DJ30 41 3 94 CPFES[...]]VRQVEK A0A151DJ30 55 119 140 PWSGD[...]NWPTYKD A0A0L0D2B9 872 74 326 MPPRV[...]TTKWSKK V8NIV9 919 547 648 SFKYL[...]LEAKEC A0A1D2MNM4 927 13 109 GTRVW[...]IYTYCG A0A1D2MNM4 999 119 437 PWSGDN[...]]RQDTVT A0A167EE16 1085 167 236 KTYLS[...]YELLTT A0A0A0M635 1104 189 269 KADQE[...]INLVIV Since I know repeaters also creates additional overhead I am doing all my benchmarks without them. I can always build the websites without them. In the next days I will do some benchmarks including repeaters just to see how it goes. Once again, thanks for all the replies!

Unless I'm forgetting something, the $pages->uncache($page); won't help here because $page is a newly created Page that wasn't loaded from the database. So it's not going to be cached either. Uncaching pages is potentially useful when iterating through large groups of existing pages. For instance, if you are rendering or exporting something large from the contents of existing pages, you might like to $pages->uncacheAll() after getting through a thousand of them to clear room for another paginated batch. Though nowadays we have $pages->findMany() and $pages->findRaw(), so there are fewer instances were you would even need to use uncache or uncacheAll, if ever. ProcessWire actually does an uncacheAll() internally after saving a page already. This is necessary because changes to a page or additions/deletions to the page tree may affect other pages, and we don't want any potential for old cached data to appear in future $pages->find() or other operations. Just one example is if we called $parent->children() before a save, and then after the save called it again, we'd want our new page to be in the children rather than having it return the previously cached value. There are a lot of similar cases, so the safest bet is for PW to uncache the results of future page get/find operations after a save as the default behavior. So that's the way it's always done it. As far as I can tell from fedeb's example (and often other with import operations), it may be better to tell PW to skip this "uncacheAll-after-save" behavior. That's because imports often involve Page reference fields, and you don't want PW to have to reload referenced pages after every save. So you could potentially reduce overhead by telling it not to uncache after save, i.e. $pages->save($page, [ 'uncacheAll' => false ]); I'm not sure if fedeb's import involves loading of any other pages, whether for page reference fields, or anything else. So it may not matter one way or the other here, but wanted to mention it just in case. I know about ProcessWire tuning, but not about MySQL server tuning. When dealing with 20 million rows that seems like getting into the territory where optimizations to the DB configuration deserve a lot of focus, so I would bet that BitPoet's suggestions are going to make the most difference.

You mean something like the screenshot attached? That's possible with some CSS (this is just an example, how one could do this) .show-banner > .privacywire-page-wrapper, .show-options > .privacywire-page-wrapper, .show-message > .privacywire-page-wrapper { position: fixed; left: 0; top: 0; right: 0; bottom: 0; background: rgba(0, 0, 0, 0.6); backdrop-filter: blur(5px); display: flex; justify-content: center; align-items: center; } .privacywire { display: none; } .show-banner .privacywire.privacywire-banner, .show-options .privacywire.privacywire-options, .show-message .privacywire.privacywire-message { display: block; position: relative; left: auto; right: auto; bottom: auto; max-width: 450px; }

I think I don't understand your question.. I don't have any functions in init.php nowadays. I have everything in init() of a module or in init() of a pageclass. If there is something in init() of a pageclass I have to trigger that manually, which is the reason for the loadClasses method of RM which triggers init() automatically https://github.com/BernhardBaumrock/RockMigrations/blob/2d85e460ce5aa394480906ad5b41cd1a0e86d0fe/RockMigrations.module.php#L1991-L2003 This is a simple pageClass I'm using in my CRM for managing contacts: <?php namespace RockCRM; use ProcessWire\HookEvent; use ProcessWire\Page; class Contact extends Page { const tags = RockCRM::tags; const tpl = RockCRM::prefix."contact"; const prefix = RockCRM::prefix."contact_"; const field_skills = self::prefix."skills"; public function __construct() { parent::__construct(); $this->parent = $this->wire->rockcrm->contacts(); $this->template = $this->wire->templates->get(self::tpl); } public function init() { $tpl = "template=".self::tpl; $this->wire->addHookAfter("ProcessPageEdit::buildForm", $this, "buildForm"); $this->wire->addHookAfter("Pages::saveReady($tpl,id=0)", $this, "onCreate"); } /** * Page edit screen */ public function buildForm(HookEvent $event) { $form = $event->return; $page = $event->process->getPage(); if(!$page instanceof self) return; $crm = $this->wire->rockcrm; /** @var RockCRM $crm */ $rm = $crm->rm(); // modify page edit screen of a contact if($f = $form->get('title')) { $f->label = 'Name of the Contact'; $f->notes = 'Enter a great Name'; } // or load scripts with the new update $rm->addScript(__DIR__."/Contact.js"); } /** * Migrate this pageclass */ public function migrate() { $crm = $this->wire->rockcrm; /** @var RockCRM $crm */ $rm = $crm->rm(); $rm->migrate([ 'fields' => [ self::field_skills => [ 'type' => 'text', 'tags' => self::tags, ], ], 'templates' => [ self::tpl => [ 'tags' => self::tags, 'icon' => 'user-o', 'noSettings' => 1, 'pageClass' => '\RockCRM\Contact', 'fields' => [ 'title', self::field_skills, ], ], ], ]); } /** * Things to do when a contact is created * @return void */ public function onCreate(HookEvent $event) { $page = $event->arguments(0); $page->status = 1; // published, non-temp $event->pages->names()->uniqueRandomPageName(); // unique name } } This setup is extremely powerful, extremely clear and extremely nice to work with ? Was that what you were asking for? ?

Hi @bernhard. Prompted by your comments here , I decided to give this module a go as it looks very useful. One thing I find rather confusing is that you seem to have changed the suggested methodology and it's not entirely clear to me what is now the suggested best practice for using the module. From what I can see, you are suggesting that users construct their own module (along the lines of your example). Installing this will then run the migration and uninstalling it will do a downgrade. For further migrations, it seems that you are suggesting just adding to this module, not creating a new module. Your previous approach seems to have been more version-based. I am trying to understand how that works in practice. I currently have 2 migrations that I want to carry out. The dev system reflects the end state of both. I want to do it in 2 stages so that I can throroughly test the first migration before applying the second (although I think there is no dependency, it is better to do them in the same order as on the dev system). My plan is therefore to create the first migration module, back up the dev database, load the live database to the dev system, install the migration module to this, then test. If that is OK, then, if I understand correctly, I should amend the migration module to include the second migration and refresh the modules to run it. I would then test again. Now, what if there is a problem with this second migration and I want to reverse it? Uninstalling the module would presumably undo both migrations (unless I change the uninstall method to be only partial, which seems inconsistent). Sure, in the dev system, I could just revert to a backup, but that would not be a solution if the problems happened in the live environment as (unknown) data will also have been updated - in that case I would need to just be able to reverse the last migration. So should I put the second migration in a separate module? In which case, this starts to look more like your initial approach (except that the use of the migrate() method allows for a more declarative style). Am I clear in explaining my confusion? If so, can you de-confuse me? By way of background, here is my first migration plan (as a set of human instructions), to give you an idea about what I am trying to do: 1. Add new fields to templates: a. Property -> operator b. Property -> website2 (new field req’d) c. BookingForm -> propertyPage d. home -> taxYear, dayMonth, month & website2 (with descriptions) 2. Amend home template for a. description of operator (& webmaster?) & website (now allowed) b. re-use of summary field as restricted access field with description/instructions (and resize) c. re-order fields as per dev system 3. Change title of home to “System” 4. Add/amend hanna codes for a. ManagementMember b. Operator 5. Edit cheque payment method to = that on the dev system 6. Add images and files pages to home 7. Change title of home 8. Re-purpose comparison field as “mailHeaders” and add to StandardMails template 9. Remove bodyTop field from StandardMails template – change body to 10 rows, summary to 3, footer to 2, update descriptions & notes in the template and update instructions in the page I think I can see that there are methods for most of that in your module, but I'm not sure about the hanna codes. The php in all my hanna codes is identical (as it just passes data to a separate script), but obviously attributes etc. are different. I don't think RockMigrations handles these, but they have convenient export/import features so that's not too much of an issue. The second migration is 1. Use findPages to detect all pages with BookingStatus in a body field (i.e bodyTop etc. as well as body – any field that can take a hanna code) – approx. 15 pages. Make appropriate changes. 2. Change the refNumber for cancelled statuses (2 pages). I can probably turn these human instructions into php (although it may be quicker just to do it in the admin, but then I definitely do not have a quick 'downgrade option!).

I'm not quite ready to move any existing sites to 8, but I will be developing all new sites on 8. I haven't seen any PW core showstoppers yet and I am happy modifying any 3rd party modules if the need arises (along with a PR of the fix).

@Macrura Thanks for pointing this out. I couldn't find a documentation, about toolbar items available within processwire. I tried all of the CKEditor fullpackage and all items are usable. Here a complete list: Source, Save,NewPage, Preview, Print, Templates Cut, Copy, Paste, PasteText, PasteFromWord, -, Undo, Redo Find, Replace, -, SelectAll, -, Scayt Form, Checkbox, Radio, TextField, Textarea, Select, Button, ImageButton, HiddenField Bold, Italic, Underline, Strike, Subscript, Superscript, -, RemoveFormat NumberedList, BulletedList, -, Outdent, Indent, -, Blockquote, CreateDiv, -, JustifyLeft, JustifyCenter, JustifyRight, JustifyBlock, -, BidiLtr, BidiRtl, Language Link, Unlink, Anchor Image, Flash, Table, HorizontalRule, Smiley, SpecialChar, PageBreak, Iframe Styles, Format, Font, FontSize TextColor, BGColor Maximize, ShowBlocks About

@benbyf We ran into an issue with image resizing not working correctly on PHP 8, in particular when using focus points: https://github.com/processwire/processwire-issues/issues/1351 I'd consider this a showstopper since image resizing is absolutely necessary for responsive images, and when we tested a site on PHP 8, focus points either didn't work at all or result in fatal exceptions altogether.