Andi

Always helps to know what you're looking for ?

https://modules.processwire.com/modules/textformatter-normalize-utf8/

Thanks for the pointers @interrobang, this is going to help a lot..

4 minutes ago, interrobang said:

Btw, Wordpress is discussing this now for 6 years: https://core.trac.wordpress.org/ticket/30130

Haha ok looks like you're right on the money ?

Awesome stuff, that really makes this all seem a little less nonsensical..

6 minutes ago, interrobang said:

I am not sure if just saving is enough or if the fields need some changes tracked. Better test before re-saving 1000s of pages.

I'll absolutely make sure of that.. Will set this up tomorrow first thing in the morning and report back.

Thank you all for your help and advice!

Schönen Feierabend! ?

Hey @interrobang,

thanks a bunch. A different angle altogether.. And I guess I'll need to read up on utf normalization a little bit ?

https://stackoverflow.com/a/7934397

Quote

Unicode includes multiple ways to encode some characters, most notably accented characters. Canonical normalization changes the code points into a canonical encoding form.

Guess the problem is that I don't actually, on a technical level, understand what's happening at all.. But that does sound like we're headed in the right direction here.. I'll set up a testbed tomorrow on my localhost and try this approach, since we've already pretty much ironed out all the problematic database records on the dev site.

So am I understanding correctly that this hook would also utf-normalize textarea fields for existing records, as long as a user opened the page for editing and just hit save once?

Thanks again and greetings from Regensburg ?

1 minute ago, kongondo said:

...that Microsoft products are superior to Apple products ? ?.

...I'll crawl back under my rock now...

Haha ok then maybe it should also be mentioned that on both of my Linux machines none of this cr#p is any issue at all ?

Haven't gotten to the coding part yet, but we had a longer team conference this morning and tried to narrow down this issue a little bit.. In the name of science, so to speak.

We tried 4 people with 4 different setups, all copy/pasting to PW from the same PDF file, and those were the results:

#1 macOS High Sierra 10.13.6
- Apple PDF Preview -> PW: Broken diacritics
- Acrobat Reader -> PW: Broken diacritics
- Google Drive (Browser) internal document viewer -> PW : Good diacritics

#2 macOS Catalina 10.15.4
- Acrobat Reader -> PW: Good diacritics
- Acrobat Reader -> Text Editor -> PW: Broken diacritics (no idea why..)
- Google Drive (Browser) internal document viewer -> PW : Good diacritics

#3 Windows 10
- No Problems with diacritics during copy/paste

#4 Windows 7
- No Problems with diacritics during copy/paste

So although I'll still have to implement a hook to deal with the existing records in the database, I think we've established a workflow for #1 to get clean data into the system right off the bat, or at least for the time being.

@bernhard how did I totally overlook that all this time? ?

So if I wanted to use this on more than one fieldtype on a multilingual site I'd just need two nested foreach-loops in

// Something along these lines?
public function replaceBadUmlauts(HookEvent $event) {
    $page = $event->arguments(0);
    $bad = array("ü","ö","ä","Ü","Ä","Ö");
    $good = array("ü","ö","ä","Ü","Ä","Ö");
	$fields = ?? somehow get fields into an array;
	foreach ( ??fields as $field) {
		foreach ( ??languages as $language) {
			$page->??field = str_replace($bad, $good, $page->??field);
		}
	}
}

I'll need to dig into the documentation a little bit and see how to do this.. Thanks for putting me on the track!

Haha thanks @bernhard, you're the man. Think I'm ready to be a PW module coder just yet..?

Still a little scared of hooks although it is getting better ?

This looks promising, I'll get on it after breakfast ?

Part III

So the questions would of course then be...

1. What the flying f&#k is this sh#t ?

2. how would I approach getting all this into a hook (I'm thinking on saveReady maybe?) so this would be fixed automatically for summary & body for both language versions of the page - whenever the user pastes another crazy mess of trematized Umlaut madness into an artists profile? ?

Thank you all in advance & god bless.. Glad I finally got this off my chest

Part II

So one year later we're working a site that features little artist profiles with a summary, body/longer band info, and some pictures. These guys are from all over the place, Spain, France, Germany and all the docs the client is copy/pasting from are PDFs. He's also on a Mac (if that's at all related), and the ü's are back

Frontend:

Backend:

Together with of tons of messed up diacritics in Spanish & French names and so on. Also this time both lower (äöü) and uppercase (ÄÖÜ) have tremas, all of them.

So no way we could manually replace all that for currently ~250 artists in both the English and German language versions. We did try pasting the text into a Code Editor first and then copying it over to PW, but that also did nothing..

So the workaround we're currently using is this:

This is what the bookmark in Finder looks like that helped me build a selector to get the affected pages. I just copy/pasted the "bad" Umlauts straight from PWs backend fields into the value fields here.

And then wrote a makro in Tracy Debugger which currently looks like this

// CAREFUL WITH $PAGE IN MAKROS

$pgs = $pages->find("summary%=ü|ö|ä");
bd($pgs);
bd("Found " . $pgs->count() . " pages");

// try to add something to this array in Tracy or a code editor of your choice and watch all hell break loose
$bad = array("ü","ö","ä","Ü","Ä","Ö");
bd($bad);

$good = array("ü","ö","ä","Ü","Ä","Ö");
bd($good);

foreach($pgs as $p) {
    $p->of(false);
    $summary_old=$p->summary;
    $summary_new = str_replace($bad, $good, $summary_old);
    bd($summary_new);
    $p->summary = $summary_new;
    $p->save('summary');
}

Which, on it's face, looks like the most ridiculous piece of code you could ever write. But it works ?

So here's a thing I've been chewing on for a while now.. And although I kind of have a handle on it at this point, I'd still love to understand what is actually happening here ?

Please bear with me.. Last year I did a project for a client in Lübeck, Germany, and implemented a full text search on $page->body and $page->summary not too different from @ryan's default site frontend search functionality.

Got it all to work nicely without too much hassle, client was already filling up the site with content, and then one day I decided to test the frontend search by looking for pages with "Lübeck" in them. Which threw out 3 matches in more than 50 pages. Naturally, with the site being in Lübeck, that number seemed a little off, so I checked manually and realized there were lots of pages missing from the results. So I thought ok something's broken about my search.

Half a day of testing later, and ending up using Firefox's Ctrl+F search with "Match Diacritics" + "Highlight All" on the frontend, i realized that not all ü's were the same across the site. Some would get highlighted (those that PW's search would pick up), and some wouldn't.

// try to CTRL+F this page and search for one of these
ü != ü

Talking to the client it turned out that he had copy-pasted large chunks of the content from PDF files. He was working on a Mac, so I did some research and found a bunch of information on all sorts of weird diacritics problems when doing exactly that ?

These are in german but I think the basic idea should come across

https://blog.k-webs.ch/2017-02-verschobene-punkte-ueber-umlauten.html

https://www.macuser.de/threads/falsche-umlautdarstellung-ue-punkte-versetzt.748967/

So it turned out some of the buggers were actual ü's, and some were actual u's with a "trema" attached to them.  Something a little bit like this

// but not quite
u¨

Now the part where this gets even more confusing is, on the frontend side they don't look the same across different OS's / browsers / what have you. Chrome on apple seems to handle them completely different from FF on Windows 7 for instance, so the client couldn't even understand what the problem was because he didn't see it.

Also please note that during that particular project all other Umlaute / Diacritics were fine, äs, ös, uppercase, lowercase,, everything. Just the little ü's were acting up. So we sat down and started out search & replacing them. There was not a lot of time, aaand well I was new to PW ?

... stay tuned for Part II where we're doing a project with over 200 international artists with all sorts of funky diacritics in bandinfos *all coming from PDF Files* ?

Set this up for the first time tonight, works really well over here. Just to save someone else some head-scratching, if you do this:

On 9/14/2017 at 12:30 AM, Robin S said:

If you already have some Page Reference values in place before you start using the module on those fields then you can get things set up with a one-off API operation:

// Get all pages where page_field_a is not empty
$pgs = $pages->find("page_field_a.count>0");
...

Consider adding include=all to your selector to also iterate through unpublished and hidden pages when synchronizing your relationships..

// Get all pages where page_field_a is not empty, including hidden and unpublished pages
$pgs = $pages->find("page_field_a.count>0, include=all");
...

Great work on the module @Robin S, coming in very handy ?

This is the line

https://github.com/netcarver/PW-FieldtypeTime/blob/4332e922d6c8e98b64e8e6e408ad7c71e318ffff/InputfieldTime.module#L83

So for the time being, in InputfieldTime.module change line 83 to something like

$attrs['placeholder'] = "00:00";

Awesome module @Mike Rockett, been using this on a couple of sites last year without any hiccups.

I have a case here that I'm not sure how to handle right now.

Following this post

Or this bit more specifically

/site/templates/includes/hooks.php

/**
 * This hook modifies the default behavior of the Page::path function (and thereby Page::url)
 *
 * The primary purpose is to redefine blog posts to be accessed at a URL off the root level
 * rather than under /posts/ (where they actually live). 
 *
 */
wire()->addHookBefore('Page::path', function($event) {
  $page = $event->object;
  if($page->template == 'post') {
    // ensure that pages with template 'post' live off the root rather than '/posts/'
    $event->replace = true;
    $event->return = "/$page->name/";
  }
}); 

I set up a hook to mask a url segment that I don't want to show up in page paths in a certain section of my site.

This is all fine and dandy and working as advertised, but since the hook is living in /site/templates/ and is only affecting the front-end part of the site, the paths that show up in sitemap.xml still show the original url segment ( /posts/ in @ryan 's example)

Now ideally I'd rather not mess with PW's page path behaviour in the backend, so I'm wondering if there's a way to somehow make sitemap.xml reflect the hooks from the front-end side of things without too much hacking around?

Also new to hooks and trying to be careful around them ?

All the best and thanks in advance!

Dude you're fast! ?

Looking great so far, nice and simple. With this in site/templates/_init.php I can do

// retrieve all images that don't have mytag1
$images = $page->images_header->excludeTag('mytag1');

// retrieve all images that don't have mytag2
$images = $page->images_header->excludeTag('mytag2');

// retrieve all images that have neither mytag1 or mytag2
$images = $page->images_header->excludeTag('mytag1|mytag2');

I'll test this further and report back. Awesome stuff @Jan Romero, much appreciated!!

Continuing my journey into PW hooks, I'm trying to find a way to retrieve all images from a page that explicitly *do not* have a certain tag (or tags) attached to them.

Found this post from 2015

But I'm wondering if there's a more elegant way to go about this.

Let's say I have a multi-image field called "images_header" and instead of

$page->images_header->findTag('mytag');

I would like to do this:

$page->images_header->excludeTag('mytag');

So I'd be able to do

// find images that don't have the tag "mytag"
$images = $page->images_header->excludeTag('mytag');
// check if there's any images
if (count($images)>0) {
	// do something..
}

Would this be possible by hooking into Pagefiles somehow?

There's this bit in /wire/core/Pagefiles.php Line 626 that I'd basically just need to reverse (or at least in my mind ? )

public function findTag($tag) {
	$items = $this->makeNew();		
	foreach($this as $pagefile) {
		if($pagefile->hasTag($tag)) $items->add($pagefile);
	}
	return $items; 
}

Any ideas on how this could be done in a graceful manner?

Thanks in advance!

Hey again,

following @bernhard's suggestion of making stuff a little more readable, and after some refactoring, here's the result:

In templates/admin.php

// user custom theme warning in backend
$this->addHookAfter('Process::execute', function($event) {
	// make stuff more readable
	$user = $this->wire('user');
	// early exit if user is not logged in
	if(!$user->isLoggedin()) return;
	// array of themes that will be excluded
	$x = array('', 'default');
	// early exit if current user theme is in excluded array
	if(in_array($user->custom_theme->value, $x)) return;
	// set up message
	$u = $user->name;
	$t = $user->custom_theme->title;
	// set up path for profile link
	$p = $this->wire('config')->urls->admin . "profile";
	// send message
	$this->wire->warning("Frontend wird für Benutzer \"$u\" aktuell mit dem Theme \"$t\" ausgegeben.
						<small>Einstellung \"Frontend Theme\"im <a href=\"$p\">Benutzer-Profil</a></small>", Notice::allowMarkup);
});

Also added an array of values to be excluded to make this more reusable in the future.

Looking neat, working as expected. Thanks again and servus from Bavaria!

Thanks @bernhard, it really does ?

I'm in my 5th or so project with PW but haven't really used hooks so far. Looks like the next best thing to a love affair ?

Thank you both again for your help, greatly appreciated!

Works a treat. Thanks for explaining, that was very helpful!

Here's the finished hook for future reference.

In templates/admin.php

// user template switch warning in backend
$this->addHookAfter('Process::execute', function($event) {
	// only works for logged in users that have a custom frontend template enabled
	if ($this->wire('user')->isLoggedin() && $this->wire('user')->template_switch->value != 'default') {
		$u = $this->wire('user')->name;
		$t = $this->wire('user')->template_switch->title;
		$p = $this->wire('config')->urls->admin . "profile";
		$this->wire->warning("Frontend der Seite wird für User $u aktuell mit dem Template \"$t\" ausgegeben. <small>Einstellung \"Frontend Template\"im <a href='$p'>Benutzer-Profil</a></small>", Notice::allowMarkup);
	}
});

Thanks a bunch @Jan Romero that was quick ?

Very elegant and simple. And works almost perfectly, but when I set the option from "crazy" back to "default", the message pops up one more time. When I refresh the backend or navigate to a different page in admin it's gone..

Going the other way around, from "default" to "crazy", the message pops up right away.

Nothing too major but I'm wondering how to work around that..

Getting a little deeper into the ProcessWire state-of-mind here. I seriously think I wouldn't have come back to webdev if it wasn't for this wonderful little gem of a CMS.

I have an "Options" field added to all users on a site. If the user has anything other then "default" selected, I would like to show a permanent message in the admin like the one in the screenshot, only so that the user can't close it. As a friendly reminder that he changed that option from default to something crazy ?

I've read up on how to send messages to users, but where would I hook into to make this show up all the time in the backend?

https://processwire.com/api/ref/wire/message/

Thanks in advance!

@netcarver Field is showing "PLACEHOLDER" when empty, is that intentional or is there a way to configure the placeholder value?

Also a default value option would be very, very neat. I'm running this on an event planner that has events almost always starting around the same time of day. Would that be hard to implement?

Running version 0.2.2, field is inside a repeater in this case.

Thanks a bunch!

Very cool module @kixe.

Would it be hard to implement date formatting for custom datetime fields instead of just PW's built in "date" field?

Setting up an event planner over here, where every event has a event_date field (datetime) that I'm trying to prepend to the event's url slug (or name in PW's terminology ?).

If i use "event_date title" as the name format it works, if i try "event_date(Y-m-d) title" I get an error:

Quote

Error: 'Name Format Children' in settings of template 'location'. Unknown property, field or subfield: 'event_date(Y-m-d)'

Currently working around the issue by changing the field's output formatting directly, but I was wondering if that would be a feature that might come in handy in the future.

Cheers and happy easter!

Hey @Mats,
great module! I was playing around with pages->find today and was getting matches for pages that didn't have map markers stored to them, or at least I thought. On closer inspection in MySQL I found that there were DB rows for all pages that had the template with my map marker field, not just the ones with actual markers stored.

Most of the empty records look like this

data: EMPTY lat: 0.000000 lng: 0.000000 status: -1 zoom: 0

Some like

data: EMPTY lat: 0.000000 lng: 0.000000 status: -100 zoom: 0

Which I'm assuming would be the ones that once had a marker which was later removed through "Clear"

And some like this:

data: EMPTY lat: 0.000000 lng: 0.000000 status: -100 zoom: 16

Which I guess are pages that someone just fumbled around with the zoom on in the backend ?

So in order to find pages with actual map markers right now I'd need to set up my selector like

 template=mytemplate, map_marker.lat!=0, map_marker.lng!=0 

Instead of just

map_marker!=""

This might also explain why the marker input field keeps popping up in page edit mode, even though I've set the visibilty to "closed when blank & open when populated".

Am I missing something obvious or is this unintentional behaviour?

Best regards!

On 6/3/2016 at 7:25 PM, Jonathan Lahijani said:

Another way is to insert it via JavaScript.

In case anyone should need it, this still works with the latest UiKit Theme, just replace the jQuery selector with:

$(document).ready(function(){ 
  $('#pw-masthead .pw-primary-nav').append('<li><a href="https://example.com/" target="_blank">Example</a></li>'); 
});

Or @Jonathan Lahijani's recipe for a less quick & dirty approach:

https://processwire-recipes.com/recipes/create-custom-admin-settings-page-link-in-admin-menu/

2 hours ago, Robin S said:

3. Use a saveReady hook to automatically parse titles into components and save these to dedicated sort fields. This seems like the most promising option to me.

An outline of how you could do option 3...

It seems like a long way to go, but this looks fantastic. I'll get on it asap, thanks a bunch @Robin S and @bernhard.

@marcus Wondering if this might deserve a recipe, I could imagine quite a bunch of people are struggling with this issue.

Thanks again, and greetings to NZ!