Add an allowed character to default seach form. [solved]

[TLT]

Hello, and thanks again for PW. The more I use it, the more I like it.

I have a silly problem, and since I have tested several (bad) solutions, I come and ask for help. I think I'm just being thick there...

Is there a relatively simple way to add an allowed character to the default search form's sanitizer ?

FYI, I'm working on a dictionary in breton language (60k entries), and I must find a way to allow apostrophes in search input. (In breton language, "CH" and "C'H" are actually two different letters altogether).

I must say I'm a bit confused by the sanitizer. On the regular search form, it appears three times. Once in the input field itself, and twice on the search template.

I did read the sanitizer page and sanitizer class page, but I'm not sure to understand where to go from there. From what I understand, the

$sanitizer->name 

methods would accept some extra allowed character, but I seem to have no luck in adding this to my template. Morevover, do I have to change any sanitizer call to a -> name one ?

The sanitizer php core page includes an "allowedASCII" which seems very promising, but is there a way to override it ? Hooks ? (not used to them I'm affraid...)

Morevover, the fact that i'm trying to implement the (') character worries my a bit. Can I escape it ? would something like '\'' do the trick ?

Thank in advance to you, good men. The site building is reaching an end, and I am tired indeed. (you wouldn't believe the state of the initial DB I had to work from...)

[LostKobrakai]

I don't think sanitizers will be the kicker here, but rather which characters are supported by processwire's selector engine and/or mysql's handling of those characters. I'm not sure, but $sanitizer->selectorValue might give at least some insights.

[TLT]

Well... Thank you for the tip. Might well be, however, I have been looking at selectorValue, and it actually seems to handle apostrophes. (I might be wrong here)

However, Alice would say thing are getting curiouser and curiouser... If I echo my $q and $selector on the search page, and test with a "c'hai", for instance, I do get "c'hai" printed. So the apostrophe might indeed survive the sanitizers.

However, when I try to echo a

$pagechai = $pages->get("title=c'hai");
echo "<p>test appel:{$pagechai->name}</p>";

i get nothing, even if I try a title=c\'hai

Hmmm... Maybe this has some deeper root than a simple sanitizer. Does PW clean direct calls to it's pages titles ? Thiscould be a problem in my specific case.

[adrian]

That pages find works here. Not sure why it's not working at your end. What version of PW are you running? Anything unusual with your server setup that you can think of?

[TLT]

o_O

OK. First, thanks for checking. This will definitely spare me hours of pain. Definitely from my end. (I prefer this...)

Processwire is 3.0.42. Only "weird" thing I can think of on my dev server is PHP 7.0 (I will try with 6)

Oh, and I started with a blank profile and just copied the search page from default profile when I needed it. (don't ask me...), although searches mostly work.

Must be some misconfiguration somewhere in my settings. I'll try and start fresh in the morning.

Thank you very much, I would probably have lost hours before thinking of trying on a fresh install. I'll let you know if I discover what is the cause of this.

[adrian]

I am running PHP 7, so don't worry about that.

I am running the latest PW dev 3.0.57, but I doubt there'd be an issue in 3.0.42, but maybe worth checking.

Did you go with standard MyISAM and UTF8 in your MySQL database? I don't think this should be an issue either, but just something to check.

[TLT]

UTF8 and MyISAM, yep. MySQL standard from ubuntu repos.

But you might be right, now that you mention it, I can have reasonable doubts about my SQL config. I had to do some dirty things in order to recover data from a massive pile of sugar a while ago. 

I will try to dump this on another server tomorrow. It was planned anyway. Then I'll be sure.

Thanks for the head up. I'll let you know if I have any luck.

[TLT]

OK.

Note for later : sleeping is sometimes useful.

So it eventually had nothing to do with PW search nor sanitizer. It was more about me being a moron.

This came from libreoffice not inserting the correct apostrophe character in the source data sheets. A simple search and replace in my original CSV will sort this out (I had to change a few things anyway).

Thank you all, folks, I could have lost hours before thinking of checking the source material. (as I should probably have done first....)