UTF8 pageName sanitization?

[bernhard]

yesterday i imported some blog-posts from an old pw version to a new one. worked like a charm, but today i saw that the url does not show german umlauts. the pagename is created from the title automatically after sanitization, so i tried this:

and after researching i found out about the new function: https://processwire.com/api/ref/sanitizer/page-nameut-f8/

As you see, the result is not right! my config says this:

$config->pageNameCharset = 'UTF8';
$config->pageNameWhitelist = '-_.abcdefghijklmnopqrstuvwxyz0123456789æåäßöüđжхцчшщюяàáâèéëêěìíïîõòóôøùúûůñçčćďĺľńňŕřšťýžабвгдеёзийклмнопрстуфыэęąśłżź';

and trying lowercase letter it got even worse!

am i missing anything?

pw 3.0.30

[BitPoet]

Silly question perhaps, but is your site/config.php saved as UTF8?

[bernhard]

hm... i guess yes, as it works correctly when i save a standard page with pagename äöü...

edit, yes
 

.../site# file -bi config.php
text/x-php; charset=utf-8

 

[bernhard]

just tried it on another site, so i consider this as a bug. can anybody confirm, please, then i will create an issue on github...

[BitPoet]

I only get the same incorrect output as you if the script that invokes pageNameUTF8 (or the passed string) is encoded in iso-8859 instead of utf8. When everything is properly encoded, I get back the string with umlauts intact.

[Soma]

It works fine for me all ok. In backend, via API too.

[LostKobrakai]

@bernhard are you sure it's not something in tracy's ajax request messing with the encoding? I wouldn't rely on those console tests to heavily.

[adrian]

I can confirm that it works fine here from a template file, but as @LostKobrakai there is something in the Tracy console AJAX request that is breaking the encoding. I'll try to look into it sometime today and provide a fix if I can.

[adrian]

Hi again,

Ok, the console panel in the latest version of Tracy now works fine.

When I initially coded it I was using GET not POST (I don't even remember why, but obviously a bad idea), but it meant I needed to base64 encode everything which was breaking the UTF8 of your content. Even though I switched to POST even before I released it, I never removed the base64. I could have recoded it as utf8 which actually does work, but I think the right thing to do is simply remove the base64 encoding, which has also made the console much snappier. 

Anyway, you will notice that the first time you open the console (after upgrading to the latest version) that the contents will be base64. After that you should be fine.

Please let me know if there are any other issues along these lines or any other situations where you feel you can't trust the console panel.

 

PS Sorry for the time everyone spent on this!