Issue with finding existing data in field

[Manon]

Working with a multi-language site.

I have a field 'province' with the list of provinces in Options in English and in French tab. The field is selectable in the 'municipality_page' template as a dropdown box. I have an Ajax call to retrieve the choice of the province to list the municipalities of that province.

Everything works well in English, however, I am having an issue with the French, I don't know if it is because the French provinces uses accents and dashes in them. I was able to make it work for 9 out of 12 provinces with the following code.

$province = $_GET['prov'];

if(strpos($province, "-") !== false){
    $choice = str_replace("-", "|", $province);
} else {
    $choice = $province;
}

$entries = $wire->pages->find("template=municipality_page, province%=$choice");

Just can't figure out how to get 'Colombie-Britannique', 'Nouvelle-Écosse' and 'Territoires du Nord-Ouest' to be found. I tried manually with the code below, but it didn't even worked. It also doesn't seemed to be a typo in the field.

$entries = $wire->pages->find("template=municipality_page, province%=Colombie-Britannique");

What is missing in the code to be able to find those 3 provinces?

[BitPoet]

You can use quotes around selector values, as documented here. So you might try if that solves your problem without any substitutions.

$entries = $wire->pages->find("template=municipality_page, province%=\"$choice\"");

Though, to be on the safe side, you should always sanitize all user input (the sanitizer can even add quotes where necessary):

$entries = $wire->pages->find(
  "template=municipality_page, province%=" . $sanitizer->selectorValue($choice, array("useQuotes" => true))
);

Also, make sure your file encoding (also of the "prov" GET parameter) is UTF-8.

[Manon]

Using the quotes didn't fix it, now the other provinces with dashes are not found, 'Nouveau-Brunswick', 'Terre-Neuve-et-Labrador' and 'Île-du-Prince-Édouard'. So not the solution.

Note that there is no user input, just an onclick event which triggers the ajax call.

function displayProv(prov){
$.ajax({
        url: '/ajax/getParticipants.php',
        type: 'GET',
        data: {prov},
        dataType: 'JSON',
        success: function(response){
            $("a#prov-name").attr("href", response.ProvinceUrl);
            $("a#prov-name").text(response.ProvinceChoice);
            $("#municipalities-list").html(response.MunicipalitiesList);
        }
    });
}

I had already tried adding contentType with UTF-8 but didn't do anything so I remove it.

Any other suggestions?

[BitPoet]

I'm pretty sure that at least a big part of the problem is a charset issue, as PW itself otherwise has no problem searching for accented characters. Use your browser's debug tools to check every request and response (page itself, ajax request/response) and make sure that all the templates and other files involved are also saved in UTF-8 format.

Did you try the quotes together with your pipe substituion? If yes, it would explain why the others weren't found anymore, as the pipe then was no longer recognized as an alternative match operator. province%=Nouveau|Brunswick means that a search is successful if either Nouveau or Brunswick is found in the province field, whereas the pipe character is threated like any other character inside quotes.

[Manon]

Yes, with the quotes I commented out the code for the pipe substitution and also tried without the % but with no luck.

I have the browser debug on and can see the results of request with no apparent error.

Just in case, how exactly do I check for UTF-8 format?

[BitPoet]

First, look at the page properties. Firefox e.g. shows you the text encoding when you right click and select "Properties".

Look at the headers of all requests and responses. Each response should have a Content-Type header that states the type itself and the charset used, something like:

Content-Type: text/html; charset=UTF8

If there's no charset header in the response but you have an HTML document, a meta header stating the charset is also sufficient. That meta header can have one of two forms:

<meta charset="utf-8"/>
<!-- or: -->
<meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>

In the ajax request, the "prod" get parameter should contain percent-encoded double-byte values if non-ascii-characters are present, like

/ajax/getParticipants.php?prod=Nouvelle-%C3%89cosse

If you see %C9 here instead of %C3%89, then your ajax call is submitting its data in the wrong charset. In that case, look at the HTML page (= PHP files) that loads it.

Then, sometimes just one involved file (PHP include, javascript file) is in the wrong encoding and makes PHP fall back to an 8 bit encoding. You can check this (99% reliably) from within PHP by running a small one-liner (or putting that into a script if you don't have command line access and calling it over the web):

echo mb_detect_encoding(file_get_contents("/path/to/file/you/want/to/check"));

The most likely suspect for being non-UTF-8 would be getParticipants.php, so I'd check that one first.

Btw., I would expect this line:

1 hour ago, Manon said:

        data: {prov},

 

to read:

        data: {prov: prov},

instead, but that may of course just be a copy&paste error.

[Manon]

from what I can see, it is all UTF-8.

Just don't understand why 'Nouveau-Brunswick', 'Terre-Neuve-et-Labrador' and 'Île-du-Prince-Édouard' works but not 'Colombie-Britannique', 'Nouvelle-Écosse' and 'Territoires du Nord-Ouest'.

[BitPoet]

Hm, I understood your original post so that searching for words with accented characters only works if you substitute the hyphen, but I may have misread that.

Still, trying to solve one mystery at a time: are the pages that aren't found activated for the French language? (Since Colombie-Britannique should be found in any case)

[Manon]

Yes I have checked that and they are activated.

[BitPoet]

And you can find all 12 if you run a find for template=municipality_page without a province?

[Manon]

Yes I can see all data (pages) using the template without a province selected

[Manon]

just tried with the id for 'Colombie-Britannique' and it worked so how would I get the $choice id instead?

$entries = $wire->pages->find("template=municipality_page, province=2");

 

[Robin S]

2 hours ago, Manon said:

just tried with the id for 'Colombie-Britannique' and it worked so how would I get the $choice id instead?

How are you setting the 'prov' GET parameter? You say it isn't user input so I assume you are setting it in your own code. Therefore set it to the ID value instead of the title value of the option.

[Manon]

Since I couldn't figure out why it couldn't find only those 3 provinces in the option field using the title, I decided to manually look for them and those with dashes with their corresponding id.  The others continue using the title. It's not ideal but it works.

$province = $_GET['prov'];

if($province == 'Colombie-Britannique'){
	$choice=2;
} else if($province == 'Nouveau-Brunswick'){
	$choice=4;
} else if($province == 'Terre-Neuve-et-Labrador'){
	$choice=5;
} else if($province == 'Territoires du Nord-Ouest'){
	$choice=6;
} else if($province == 'Nouvelle-Écosse'){
	$choice=7;
} else if($province == 'Île-du-Prince-Édouard'){
	$choice=9;
} else {
	$choice = $province;
}

$entries = $wire->pages->find("template=municipality_page, province=$choice");

Please advise if somebody finds a better solution.

Thank you.