Andi

@bernhard you're on fire ? Looks great but I'm currently on a self-hosted Gitea instance, and it looks like they're not planning on supporting Github Actions anytime soon.. https://github.com/go-gitea/gitea/issues/12673#issuecomment-684968986 Thanks for the heads up though, I'll keep an eye on RM!

@bernhard Trying this on a test project currently, I'm wondering if there's an easy way to add pages to RockFrontend the same way you're doing with $home. Let's say, I normally have a /contact/ page, stored in $contact, holding all possible information, phone numbers, social media links etc., and I'd like to make that page available in all rendered files. Right now, unless I'm missing something, I could use custom variables to pass to $rockfrontend->render(), but that doesn't seem to work with ->renderLayout().. The other minor issue I'm having, if you use Alfred on an empty text block, you don't get the edit links, the whole block just disappears. Other than that, very nice work, feels really good to be using Latte again ?

If you're following the steps above, don't forget (like I did earlier) to make the bash script executable.. In Step 6, after editing your post-recieve file $ cd /var/www/myproject-git/hooks $ chmod u+x post-receive Very cool guide @gebeer, I was toying around with a webhook from Gitea calling on a local php script earlier today, which runs okay, but this seems way more elegant.. And definitely easier to set up ? Thanks!

Just a quick note, if you're on Linux & Code-OSS and have trouble opening your ALFRED / Tracy links to directly edit template code (around 34:00 in the video), you need to create a separate URL handler for vscode:// type links in ~/.local/share/applications create a file code-oss-vscode-url-handler.desktop #!/usr/bin/env xdg-open [Desktop Entry] Name=Code - OSS - VSCode URL Handler Comment=Handles opening of vscode:// URLs in Code-OSS GenericName=Text Editor Exec=/usr/bin/code-oss --open-url %U Icon=code-oss Type=Application NoDisplay=true StartupNotify=true StartupWMClass=Code - OSS Categories=Utility;TextEditor;Development;IDE; MimeType=x-scheme-handler/vscode Keywords=vscode; Terminal=false Borrowed from here: https://github.com/Microsoft/vscode/issues/48528#issuecomment-414056547 And Alfred is your new best pal ?

Thanks @bernhard! Tried that but somehow Firefox keeps acting up.. See the first two screenshots attached, one of these two pops up on every page reload (no addons activated). Vs. no problems on Chromium (last screenshot). Maybe something to keep an eye on.. I'll investigate some more and maybe try this on a live server later. ### Firefox 1: ### Firefox 2: ### Chromium:

Hey @bernhard, working through some chapters of the video. Lots of good stuff in there, just DDEV and livereload alone will be major game changers for me. And that's only the first 5 minutes ? As for the module itself, everything's working nicely so far, but Firefox console is complaining that Die Verbindung zu https://pw-rockfrontend-test.ddev.site/livereload.php?secret=###secretkey123456### wurde unterbrochen, während die Seite geladen wurde. livereload.js:7:20 English: The connection to http://site/livereload.php broke up while the page was being loaded. Happens on almost every page load, Frontend and Backend. Chromium seems fine. Also, despite the error message in FF, I'm not noticing anything breaking at all. Any ideas what could be happening here? Thanks in advance and servus from Regensburg!

As much as I appreciate @bernhard's faith in my motivation (?), and as much as I'd like to contribute, that would at this point go way above my head.. I keep pushing the day where I'll finally take a dive and learn to use git/hub/lab. And I'm still struggling to find my way around these "new" development tools and procedures.. Lots of ground to cover there currently. But yeah this seems like something that would be really useful. Maybe even as some kind of default way of dealing with user input, as I currently could only think of fringe cases where someone would not want to have this enabled..

After some more testing I feel like this should be safe to implement on the production site.. The only remaining issue at this point being.. The image description input fields seem to be a different kind of beast altogether.. Is there any way to extend the hook to cover these as well?

Ok ... ->count() ? in site/ready.php /** * run all Text and Textarea input fields through UTF-8 normalization * requires justb3a's TextformatterNormalizeUtf8 module * https://modules.processwire.com/modules/textformatter-normalize-utf8/ * * @var Inputfield $inputfield * @var WireInputData $input * @var Language $language * */ $wire->addHookBefore('InputfieldTextarea::processInput, InputfieldText::processInput', function (HookEvent $event) { $inputfield = $event->object; $input = $event->arguments(0); if (wire()->modules->isInstalled("TextformatterNormalizeUtf8")) { if ($this->languages && $this->languages->count() > 1) { foreach ($this->languages as $language) { $input_var_name = $language->isDefault() ? $inputfield->name : "{$inputfield->name}__{$language->id}"; $normalize_input = $input->$input_var_name; wire()->modules->get('TextformatterNormalizeUtf8')->format($normalize_input); $input->set($input_var_name, $normalize_input); } } else { $input_var_name = $inputfield->name; $normalize_input = $input->$input_var_name; wire()->modules->get('TextformatterNormalizeUtf8')->format($normalize_input); $input->set($input_var_name, $normalize_input); } } $event->arguments(0, $input); }); This is by no means thoroughly tested.. Use at your own risk ? Does anyone see how there could be any downsides to this approach on a live PW site?

So on the input side of things.. I'm thinking, since we don't know about ..but we have @justb3a's module we could combine @interrobang's idea of hooking into InputfieldTextarea::processInput with calling the textformatter module to sort out normalization right off the bat.. This almost works ? $wire->addHookBefore('InputfieldTextarea::processInput, InputfieldText::processInput', function (HookEvent $event) { /** @var Inputfield $inputfield */ /** @var WireInputData $input */ /** @var Language $language */ $inputfield = $event->object; $input = $event->arguments(0); if (wire()->modules->isInstalled("TextformatterNormalizeUtf8")) { bd($this->languages); // ### this doesn't seem to be an array ### if ($this->languages && $this->languages->count > 1) { // ### this part never gets called ### foreach ($this->languages as $language) { bd("language loop"); $input_var_name = $language->isDefault() ? $inputfield->name : "{$inputfield->name}__{$language->id}"; $normalize_input = $input->$input_var_name; wire()->modules->get('TextformatterNormalizeUtf8')->format($normalize_input); bd("$input_var_name :: $normalize_input"); $input->set($input_var_name, $normalize_input); } } else { // ### this part works ### $input_var_name = $inputfield->name; $normalize_input = $input->$input_var_name; wire()->modules->get('TextformatterNormalizeUtf8')->format($normalize_input); $input->set($input_var_name, $normalize_input); bd("$input_var_name :: $normalize_input"); } } $event->arguments(0, $input); }); With this, utf-8 normalizing works for the default language input fields, but currently all others remain untouched. I tried looking around for a working example of how to get to the language fields, but no luck so far.. Also feeling just a tad bit twitchy about hooking in at such a deep level, but so far no problems during testing.. Does anyone have an idea where I'm going wrong here? Thanks and all the best..! Almost there I think ?

Haha @bernhard easy on me.. Still coding like it's 2005 over here.. Guess the 10 year break made me miss out on a whole lot of stuff ? Good lord I'm glad these arrays are no longer needed ? if ($modules->isInstalled("TextformatterNormalizeUtf8")) { $pgs = $pages->find("template=artist"); foreach($pgs as $p) { $p->of(false); $summary=$p->summary; bd("OLD: $summary"); $this->modules->get('TextformatterNormalizeUtf8')->format($summary); bd("NORMALIZED: $summary"); $p->summary = $summary; $p->save('summary'); } }

Just getting started here but I can already confirm that @justb3a's module completely fixes this issue on the frontend side of things.. Now I'd basically just need to find a simple way to run every text & textarea field in both language versions through the normalizer.. Actually also pagetitle fields.. Or ideally, literally every bit of text that's being saved to the database.. Or would there be any downsides to that at all? --- Module works fine on PW v3.0.148 by the way

I'm aware of that, but thanks for pointing it out @interrobang Thought I just put that here for future reference, I'm imagining many people (like me until 3 hrs ago ?) haven't even heard of the term utf normalization. That thread at the wordpress tracker is a great read btw. There's a lot there already, so I think we should be able to get this whole thing sorted out by tomorrow night. Cheers & danke vielmals ?

Always helps to know what you're looking for ? https://modules.processwire.com/modules/textformatter-normalize-utf8/ Thanks for the pointers @interrobang, this is going to help a lot..

Haha ok looks like you're right on the money ? Awesome stuff, that really makes this all seem a little less nonsensical.. I'll absolutely make sure of that.. Will set this up tomorrow first thing in the morning and report back. Thank you all for your help and advice! Schönen Feierabend! ?

Hey @interrobang, thanks a bunch. A different angle altogether.. And I guess I'll need to read up on utf normalization a little bit ? https://stackoverflow.com/a/7934397 Guess the problem is that I don't actually, on a technical level, understand what's happening at all.. But that does sound like we're headed in the right direction here.. I'll set up a testbed tomorrow on my localhost and try this approach, since we've already pretty much ironed out all the problematic database records on the dev site. So am I understanding correctly that this hook would also utf-normalize textarea fields for existing records, as long as a user opened the page for editing and just hit save once? Thanks again and greetings from Regensburg ?

Haha ok then maybe it should also be mentioned that on both of my Linux machines none of this cr#p is any issue at all ?

Haven't gotten to the coding part yet, but we had a longer team conference this morning and tried to narrow down this issue a little bit.. In the name of science, so to speak. We tried 4 people with 4 different setups, all copy/pasting to PW from the same PDF file, and those were the results: #1 macOS High Sierra 10.13.6 - Apple PDF Preview -> PW: Broken diacritics - Acrobat Reader -> PW: Broken diacritics - Google Drive (Browser) internal document viewer -> PW : Good diacritics #2 macOS Catalina 10.15.4 - Acrobat Reader -> PW: Good diacritics - Acrobat Reader -> Text Editor -> PW: Broken diacritics (no idea why..) - Google Drive (Browser) internal document viewer -> PW : Good diacritics #3 Windows 10 - No Problems with diacritics during copy/paste #4 Windows 7 - No Problems with diacritics during copy/paste So although I'll still have to implement a hook to deal with the existing records in the database, I think we've established a workflow for #1 to get clean data into the system right off the bat, or at least for the time being.

@bernhard how did I totally overlook that all this time? ? So if I wanted to use this on more than one fieldtype on a multilingual site I'd just need two nested foreach-loops in // Something along these lines? public function replaceBadUmlauts(HookEvent $event) { $page = $event->arguments(0); $bad = array("ü","ö","ä","Ü","Ä","Ö"); $good = array("ü","ö","ä","Ü","Ä","Ö"); $fields = ?? somehow get fields into an array; foreach ( ??fields as $field) { foreach ( ??languages as $language) { $page->??field = str_replace($bad, $good, $page->??field); } } } I'll need to dig into the documentation a little bit and see how to do this.. Thanks for putting me on the track!

Haha thanks @bernhard, you're the man. Think I'm ready to be a PW module coder just yet..? Still a little scared of hooks although it is getting better ? This looks promising, I'll get on it after breakfast ?

Part III So the questions would of course then be... 1. What the flying f&#k is this sh#t ? 2. how would I approach getting all this into a hook (I'm thinking on saveReady maybe?) so this would be fixed automatically for summary & body for both language versions of the page - whenever the user pastes another crazy mess of trematized Umlaut madness into an artists profile? ? Thank you all in advance & god bless.. Glad I finally got this off my chest

Part II So one year later we're working a site that features little artist profiles with a summary, body/longer band info, and some pictures. These guys are from all over the place, Spain, France, Germany and all the docs the client is copy/pasting from are PDFs. He's also on a Mac (if that's at all related), and the ü's are back Frontend: Backend: Together with of tons of messed up diacritics in Spanish & French names and so on. Also this time both lower (äöü) and uppercase (ÄÖÜ) have tremas, all of them. So no way we could manually replace all that for currently ~250 artists in both the English and German language versions. We did try pasting the text into a Code Editor first and then copying it over to PW, but that also did nothing.. So the workaround we're currently using is this: This is what the bookmark in Finder looks like that helped me build a selector to get the affected pages. I just copy/pasted the "bad" Umlauts straight from PWs backend fields into the value fields here. And then wrote a makro in Tracy Debugger which currently looks like this // CAREFUL WITH $PAGE IN MAKROS $pgs = $pages->find("summary%=ü|ö|ä"); bd($pgs); bd("Found " . $pgs->count() . " pages"); // try to add something to this array in Tracy or a code editor of your choice and watch all hell break loose $bad = array("ü","ö","ä","Ü","Ä","Ö"); bd($bad); $good = array("ü","ö","ä","Ü","Ä","Ö"); bd($good); foreach($pgs as $p) { $p->of(false); $summary_old=$p->summary; $summary_new = str_replace($bad, $good, $summary_old); bd($summary_new); $p->summary = $summary_new; $p->save('summary'); } Which, on it's face, looks like the most ridiculous piece of code you could ever write. But it works ?

So here's a thing I've been chewing on for a while now.. And although I kind of have a handle on it at this point, I'd still love to understand what is actually happening here ? Please bear with me.. Last year I did a project for a client in Lübeck, Germany, and implemented a full text search on $page->body and $page->summary not too different from @ryan's default site frontend search functionality. Got it all to work nicely without too much hassle, client was already filling up the site with content, and then one day I decided to test the frontend search by looking for pages with "Lübeck" in them. Which threw out 3 matches in more than 50 pages. Naturally, with the site being in Lübeck, that number seemed a little off, so I checked manually and realized there were lots of pages missing from the results. So I thought ok something's broken about my search. Half a day of testing later, and ending up using Firefox's Ctrl+F search with "Match Diacritics" + "Highlight All" on the frontend, i realized that not all ü's were the same across the site. Some would get highlighted (those that PW's search would pick up), and some wouldn't. // try to CTRL+F this page and search for one of these ü != ü Talking to the client it turned out that he had copy-pasted large chunks of the content from PDF files. He was working on a Mac, so I did some research and found a bunch of information on all sorts of weird diacritics problems when doing exactly that ? These are in german but I think the basic idea should come across https://blog.k-webs.ch/2017-02-verschobene-punkte-ueber-umlauten.html https://www.macuser.de/threads/falsche-umlautdarstellung-ue-punkte-versetzt.748967/ So it turned out some of the buggers were actual ü's, and some were actual u's with a "trema" attached to them. Something a little bit like this // but not quite u¨ Now the part where this gets even more confusing is, on the frontend side they don't look the same across different OS's / browsers / what have you. Chrome on apple seems to handle them completely different from FF on Windows 7 for instance, so the client couldn't even understand what the problem was because he didn't see it. Also please note that during that particular project all other Umlaute / Diacritics were fine, äs, ös, uppercase, lowercase,, everything. Just the little ü's were acting up. So we sat down and started out search & replacing them. There was not a lot of time, aaand well I was new to PW ? ... stay tuned for Part II where we're doing a project with over 200 international artists with all sorts of funky diacritics in bandinfos *all coming from PDF Files* ?

Set this up for the first time tonight, works really well over here. Just to save someone else some head-scratching, if you do this: Consider adding include=all to your selector to also iterate through unpublished and hidden pages when synchronizing your relationships.. // Get all pages where page_field_a is not empty, including hidden and unpublished pages $pgs = $pages->find("page_field_a.count>0, include=all"); ... Great work on the module @Robin S, coming in very handy ?

This is the line https://github.com/netcarver/PW-FieldtypeTime/blob/4332e922d6c8e98b64e8e6e408ad7c71e318ffff/InputfieldTime.module#L83 So for the time being, in InputfieldTime.module change line 83 to something like $attrs['placeholder'] = "00:00";