Sanitizer Transliterate

[Robin S]

A community member raised a question and I thought a new sanitizer method for the purpose would be useful, hence...

Sanitizer Transliterate

Adds a transliterate method to $sanitizer that performs character replacements as defined in the module config. The default character replacements are based on the defaults from InputfieldPageName, but with uppercase characters included too.

Usage

Install the Sanitizer Transliterate module.

Customise the character replacements in the module config as needed.

Use the sanitizer on strings like so:

$transliterated_string = $sanitizer->transliterate($string);

 

https://github.com/Toutouwai/SanitizerTransliterate
https://modules.processwire.com/modules/sanitizer-transliterate/

 

[bernhard]

Hi Robin,

thx ? I think it would be nice to also list this sanitizer in the testAll() result, so I created a PR for it: https://github.com/Toutouwai/SanitizerTransliterate/pull/2 . Unfortunately this needs a fix in the core, so I also added a PR for this: https://github.com/processwire/processwire/pull/137 .

[adrian]

Hey @bernhard - just an FYI regarding your PR which I think it a great idea by the way, but I don't think Ryan wants "all" to actually be all sanitizers:  https://github.com/processwire/processwire-issues/issues/85

Quote

excluded sanitizers are those that require secondary/non-value arguments, like match(), option() and options().

 

[adrian]

Hey @Robin S - this looks great!

I just noticed that the contents of your $default_replacements array is much more comprehensive than the list of character replacements in the InputfieldPageName settings in the core. Not sure whether Ryan would be willing to change those, but it seems to me that your list should be used there. Then perhaps you could pull that list in for your module and users of your module would benefit from any manual additions they have made to those settings, rather than using the character_replacements_str config setting in this module - it seems to me that users may need to add their custom replacements to both places at the moment.

[tpr]

1 hour ago, adrian said:

your $default_replacements array is much more comprehensive than the list of character replacements in the InputfieldPageName settings in the core

It would be great having "ő", "Ő" (=> "o") and "ű", "Ű" (=> "u") on the list. I've requested it years ago but it haven't found its way to the core.

[Robin S]

10 hours ago, bernhard said:

I think it would be nice to also list this sanitizer in the testAll()

Sure, I will keep an eye on the core and merge your PR as soon as Ryan adds support for it.

8 hours ago, adrian said:

I just noticed that the contents of your $default_replacements array is much more comprehensive than the list of character replacements in the InputfieldPageName settings in the core.

It's actually not that much more comprehensive. ?
The only additions are the macronised vowels (āēīōū), which is something I wanted for my New Zealand context because they are used in Maori. The difference is that the list includes uppercase versions of the characters. I simply took the character list from InputfieldPageName, added the macronised vowels, and ran it through mb_strtoupper(). Adding uppercase characters to InputfieldPageName wouldn't make sense because page names are forced to lowercase.

8 hours ago, adrian said:

it seems to me that users may need to add their custom replacements to both places at the moment

I had in mind that this sanitizer could be used for broader purposes besides converting characters to ASCII. Using the character replacements from InputfieldPageName seemed like a good starting point (particularly given the question that gave rise to the module) but a person could configure all kinds of replacements to suit their needs. There could well be cases where these substitutions would not be wanted in InputfieldPageName.

[adrian]

1 hour ago, Robin S said:

It's actually not that much more comprehensive. ?

Sorry, I didn't notice that it was mostly uppercase variants that were new - I just saw a lot more options ?

[mel47]

Hi!
I wondering if we transliterate any type of field. In fact, I want to transliterate titles from a fieldtypeOptions. I didn't work, however it's works well for a text field.

Thanks!

[Robin S]

@mel47, this sanitizer method will work with any text - it doesn't matter where the text comes from. Use Tracy Debugger to check that the value you are supplying to the sanitizer method is definitely text and not something else.