Warning about Server Locale after update from 3.0.52 > 3.0.53 - Help!

[EyeDentify]

Hello All PW gurus.

I just wanted your help with how i should handle the following problem.

I just recently (15 min ago) updated my website from 3.0.52 to 3.0.53, and in my admin i get a Warning:

Quote

Warning: your server locale is undefined and may cause issues. Please add this to /site/config.php file (adjust “en_US.UTF-8” as needed): setlocale(LC_ALL,'en_US.UTF-8');

My server is in Sweden, and does that mean i should set a Swedish locale in my Config.php or should it be what the warning says?

This warning gets me a little worried that something is going to break?

Thank you for all your help.

p.s: I have never before gotten this warning, just now after the update.

/EyeDentify
 

[matjazp]

https://github.com/processwire/processwire-issues/issues/184

If your server has Sweden locale installed (check with locale -a in shell), then you can use that, if not, try with what warning says. If you use ML put setlocale() in /site/init.php 

[EyeDentify]

1 hour ago, matjazp said:

https://github.com/processwire/processwire-issues/issues/184

If your server has Sweden locale installed (check with locale -a in shell), then you can use that, if not, try with what warning says. If you use ML put setlocale() in /site/init.php 

 

I Thank you for the answer.

I was wondering why this warning comes up now that i update to 3.0.53 when i before that never had the same warning?

So if i change to swedish locale in the top of my Config.php, what could happen? could i screw something up with my site?

[matjazp]

If you would upload an image named "Ö.jpg", upload would fail. setlocale() solves this issue. This warning is echoed just in PW 3.0.53. You won't screw up your site, but it's possible that date/currency (and other locale aware) functions would produce different output. 

[EyeDentify]

Just now, matjazp said:

If you would upload an image named "Ö.jpg", upload would fail. setlocale() solves this issue. This warning is echoed just in PW 3.0.53. You won't screw up your site, but it's possible that date/currency (and other locale aware) functions would produce different output. 

 

Oki  i see now.

That makes sense. Thank you for your help.

So if i set the locale in my config.php it will work for my whole PW site then ?

Where is the best place? my config.php or init.php in my site dir?

[chrizz]

I'd add it to the config just because to me a locale is like a config value. init.php might be a little late because auto load modules are already initialized

https://processwire.com/blog/posts/processwire-2.6.7-core-updates-and-more/

[adrian]

FYI - I had to add it to init.php on certain servers:

https://github.com/processwire/processwire-issues/issues/184#issuecomment-280079855

 

[matjazp]

As adrian pointed out, in multilanguage environment you have to put setlocale() in init.php, ready.php or admin.php since language support module will overwrite locale settings in config.php. Still waiting for response from Ryan about that.

[EyeDentify]

Hello Everyone. I Would like to thank everyone that pitch in on this question and try to help me.

I would like to wait for a definitive answer from Ryan on where he thinks one should put this locale setting.

If anyone of you has connections to Ryan and could ask him to check this post and give his insight to what i should do, it would be very appreciated. 

[modifiedcontent]

I also get this error and adding that line to config.php doesn't get rid of it. Where in the config.php file should I add it? Does the location matter?

I have also added it at the top of my functions.php file. Makes no difference either.

Where is init.php? Aren't those other files core files that you are not supposed to edit?

[matjazp]

Location of setlocale() in /site/config.php is not important. What locales are installed on the system? What locale are you trying to set? PW version you are using? Please upgrade to latest dev (3.0.55). Is Language Support module installed? init.php is located at /site/ folder, create it if needed.

 

[EyeDentify]

15 hours ago, matjazp said:

Location of setlocale() in /site/config.php is not important. What locales are installed on the system? What locale are you trying to set? PW version you are using? Please upgrade to latest dev (3.0.55). Is Language Support module installed? init.php is located at /site/ folder, create it if needed.

 

 

 

Hello Again Matjazp.

Thank you for the input.

So to summarize:
1. i Should put the set_locale() setting in my init.php file?

2. Should i install language support module even if I do not use multi-language fields?

3. What setting should i use for Swedish?, or should i stick with US UTF-8?

4. Is there some easy php code i could run to get a list of installed locales?

 

Thank you again for all your help.

Edited March 16, 2017 by EyeDentify
PS. i am running PW 3.0.55 at the point of posting this.

[matjazp]

1. No. 

2. Why would you install language support if you don't need it?

3. sv_SE.UTF-8

4. echo exec ('locale -a');

Have to go now, will post more information/clarification of my brief answers later, stay tuned.

[EyeDentify]

1 minute ago, matjazp said:

1. No. 

2. Why would you install language support if you don't need it?

3. sv_SE.UTF-8

4. echo exec ('locale -a');

Have to go now, will post more information/clarification of my brief answers later, stay tuned.

 

Thank You. Looking forward to the answers.

 

 

[matjazp]

First, some background to lit a light. There are some functions in PHP that may produce unexpected results in special situations. This functions are used by PW core files and in user contributed modules. Some users may have problems with file upload (pdf, zip, image, whatever) if filename has non-ascii character at the first place. For example, if user upload a file named "år.jpg" (year in Swedish language), it is expected that PW would transliterate (transform) the filename to "ar.jpg", but because of the bug in basename() PHP function, the filename would become just "r.jpg". Nevertheless the file would upload successfully. But if the filename is "привет.jpg" (Hi in Russian language) the upload would fail.

There are two ways how to handle this:

1. Write and use custom functions instead of builtin functions
2. Let end user fix that with proper locale

Option 1. is used in some CMS/CMF (like Drupal), PW (Ryan) opted for option 2. After successful login PW perform a test and warning is issued in case of test failure. It is now your responsibility to set proper system locale. The locale is a model and definition of a native-language environment. A locale defines its code sets, date and time formatting conventions, string conversions, monetary conventions, decimal formatting conventions, and collation (sort) order. Those "problematic" builtin PHP functions are locale aware, that means they work as expected, if locale is configured appropriately. 

Locale can be set by the underlying operating system or in PHP. Because on shared hosting you don't usually have root access to the OS, the only option to set the locale is by using the builtin setlocale() PHP function. But, before you can use/set the locale, it must first be installed on the OS. On most unix systems at least some basic locales are already there, they are just not used. To check what locale are you currently using and what locales are available for you to use, create the file testlocale.php at the root of your website with the following content:

<?php
    echo "<pre>Current locale:\n"
    var_dump(setlocale(LC_ALL, '0'));
    echo "\n\nAvailable locales:\n";
    echo exec('locale -a'); 

Then point your browser to http://yourwebsite/testlocale.php

If your current locale is "C" (and you are on unix), then you will probably get warning after login to the PW admin. What you want to do is change the locale to something that has "UTF-8" or "utf8" in the name of the available locales. In your case you would be looking for something like "sv_SE.UTF-8" or "sv_SE.utf8". Now, there is a chance that Swedish locale is not installed. You can ask your hosting provider to install it for you or use some other UTF-8 locale. On most unix systems I have seen, at least "en_US.UTF-8" or "en_US.utf8" is installed. If even that English locale is not available, then look for "C.UTF-8" or "C.utf8". And this is what you have to use as parameter in the setlocale() call:

setlocale(LC_ALL, "sv_SE.utf8");

You could actually "stack up" multiple locales and one will eventually work:

setlocale(LC_ALL, "sv_SE.utf8", "sv_SE.UTF-8", "en_US.UTF-8", "en_US.utf8", "C.UTF-8", "C.utf8");

Now, regarding where to put that setlocale() call. If you are not using PW multi language capabilities (you don't have Languge Support module installed), then you place setlocale to /site/config.php. That's it. If Language Support module is installed (PW checks for that), you have two options:

1. Translate the string "C" in LanguageSupport.module for each installed language
2. Put setlocale() call in /site/init.php

My preferred method is option 1 and this is what PW (Ryan) recommends. See https://processwire.com/api/multi-language-support/code-i18n/ for more info on translating files. Option 1 allows you to set different locale for each of the installed language, while  option 2 allows setting of one locale for all languages. PW will also provide links for the files that needs to be translated in the warning message.

Ryan just pushed an update that makes it use the locale setting on the frontend when using LanguageSupportPageNames module (until now just language was changed, but not locale). Also setLocale() and getLocale() methods are added to the $languages API var. 

Edited March 16, 2017 by matjazp
Info on locale on frontend

[EyeDentify]

1 hour ago, matjazp said:

First, some background to lit a light. There are some functions in PHP that may produce unexpected results in special situations. This functions are used by PW core files and in user contributed modules. Some users may have problems with file upload (pdf, zip, image, whatever) if filename has non-ascii character at the first place. For example, if user upload a file named "år.jpg" (year in Swedish language), it is expected that PW would transliterate (transform) the filename to "ar.jpg", but because of the bug in basename() PHP function, the filename would become just "r.jpg". Nevertheless the file would upload successfully. But if the filename is "привет.jpg" (Hi in Russian language) the upload would fail.

There are two ways how to handle this:

1. Write and use custom functions instead of builtin functions
2. Let end user fix that with proper locale

Option 1. is used in some CMS/CMF (like Drupal), PW (Ryan) opted for option 2. After successful login PW perform a test and warning is issued in case of test failure. It is now your responsibility to set proper system locale. The locale is a model and definition of a native-language environment. A locale defines its code sets, date and time formatting conventions, string conversions, monetary conventions, decimal formatting conventions, and collation (sort) order. Those "problematic" builtin PHP functions are locale aware, that means they work as expected, if locale is configured appropriately. 

Locale can be set by the underlying operating system or in PHP. Because on shared hosting you don't usually have root access to the OS, the only option to set the locale is by using the builtin setlocale() PHP function. But, before you can use/set the locale, it must first be installed on the OS. On most unix systems at least some basic locales are already there, they are just not used. To check what locale are you currently using and what locales are available for you to use, create the file testlocale.php at the root of your website with the following content:

<?php
    echo "<pre>Current locale:\n"
    var_dump(setlocale(LC_ALL, '0'));
    echo "\n\nAvailable locales:\n";
    echo exec('locale -a'); 

Then point your browser to http://yourwebsite/testlocale.php

If your current locale is "C" (and you are on unix), then you will probably get warning after login to the PW admin. What you want to do is change the locale to something that has "UTF-8" or "utf8" in the name of the available locales. In your case you would be looking for something like "sv_SE.UTF-8" or "sv_SE.utf8". Now, there is a chance that Swedish locale is not installed. You can ask your hosting provider to install it for you or use some other UTF-8 locale. On most unix systems I have seen, at least "en_US.UTF-8" or "en_US.utf8" is installed. If even that English locale is not available, then look for "C.UTF-8" or "C.utf8". And this is what you have to use as parameter in the setlocale() call:

setlocale(LC_ALL, "sv_SE.utf8");

You could actually "stack up" multiple locales and one will eventually work:

setlocale(LC_ALL, "sv_SE.utf8", "sv_SE.UTF-8", "en_US.UTF-8", "en_US.utf8", "C.UTF-8", "C.utf8");

Now, regarding where to put that setlocale() call. If you are not using PW multi language capabilities (you don't have Languge Support module installed), then you place setlocale to /site/config.php. That's it. If Language Support module is installed (PW checks for that), you have two options:

1. Translate the string "C" in LanguageSupport.module for each installed language
2. Put setlocale() call in /site/init.php

My preferred method is option 1 and this is what PW (Ryan) recommends. See https://processwire.com/api/multi-language-support/code-i18n/ for more info on translating files. Option 1 allows you to set different locale for each of the installed language, while  option 2 allows setting of one locale for all languages. PW will also provide links for the files that needs to be translated in the warning message.

 

Matjazp Thank you so much for giving me these answers.
And explaining everything why this is.

I think i get it now

I hope this thread gets pinned by moderators so others can get some use of the info in here.

Again, thank you for your efforts. Its greatly appreciated.

I know a lot of PW but some things still elude me, and i am looking forward to learning more as time goes on.
I love PW and i feel this is the CMF for me.

 

[cstevensjr]

A Followup

This was an interesting conversation by all parties.  What I didn't get was what actual syntax that was put in the /site/config.php file.  While this may not matter to more experienced users, it makes a difference to others.  My sites don't use Multi-Language and I was still confronted with the error that @EyeDentify noted in the original post when updating an existing site to ProcessWire 3.0.55

This is what I put in the /site/config.php file to make error go away

 

/**
* Installer: SetLocale Setting
*
* This is now needed since ProcessWire 3.0.53 to set local information
*
*/

setlocale(LC_ALL, "en_US.utf8");

As @matjazp stated, it doesn't really matter where in the file this goes.  What was confusing to me is that most other settings in the /site/config.php file started with $config and it wasn't clear whether this setting needed to follow that style.  As I said, it may not be important to experienced PHP users, but knowing what to put when confronted with this warning is helpful.

As an aside,  it would also be helpful to know the exact syntax that is needed if using the init or ready files.  Maybe I have missed something in reading all this that should have been clear to me and excuse me if I have.

If what I have used is incorrect, I ask that someone provide the correct format (styling).  Thanks.

[EyeDentify]

20 hours ago, cstevensjr said:

A Followup

This was an interesting conversation by all parties.  What I didn't get was what actual syntax that was put in the /site/config.php file.  While this may not matter to more experienced users, it makes a difference to others.  My sites don't use Multi-Language and I was still confronted with the error that @EyeDentify noted in the original post when updating an existing site to ProcessWire 3.0.55

This is what I put in the /site/config.php file to make error go away

 

/**
* Installer: SetLocale Setting
*
* This is now needed since ProcessWire 3.0.53 to set local information
*
*/

setlocale(LC_ALL, "en_US.utf8");

As @matjazp stated, it doesn't really matter where in the file this goes.  What was confusing to me is that most other settings in the /site/config.php file started with $config and it wasn't clear whether this setting needed to follow that style.  As I said, it may not be important to experienced PHP users, but knowing what to put when confronted with this warning is helpful.

As an aside,  it would also be helpful to know the exact syntax that is needed if using the init or ready files.  Maybe I have missed something in reading all this that should have been clear to me and excuse me if I have.

If what I have used is incorrect, I ask that someone provide the correct format (styling).  Thanks.

 

Hello cstevensjr. I Agree that this was very interesting.

 

I ended up using this line at the beginning of my config.php file of my sites before any config options.

setlocale(LC_ALL, "sv_SE.utf8", "sv_SE.UTF-8", "en_US.UTF-8", "en_US.utf8", "C.UTF-8", "C.utf8");

That made the error message disappear for me and has not come back.

As matjzap explained, this makes PHP try all the locales I designated until it finds one that is installed.

Then it will use that first locale that it finds.

I would like to again thank everyone in the PW community that helps me understand this problem and solve it. You are the best.

/EyeDentify

[ridgedale]

Just run into this problem following a reinstallation using the latest dev version 3.0.58. When I setup the testlocale.php file to check the current configuration:

<?php
    echo "<pre>Current locale:\n"
    var_dump(setlocale(LC_ALL, '0'));
    echo "\n\nAvailable locales:\n";
    echo exec('locale -a');
?>

and try to load the file in the browser I get the following error message:

Quote

Parse error: syntax error, unexpected 'var_dump' (T_STRING), expecting ',' or ';' in /home/xxxxx/public_html/testlocale.php on line 3

How do I go about resolving that?

In addition, I am guessing the installation is picking up that the server is not located in the US but in the UK. So should the setlocale configuration in this case simply be as follows?

setlocale(LC_ALL, "en_GB.utf8");

Another setting under section 2 of the config.php configures the timezone as:

$config->timezone = 'America/New_York';

Does the setlocale instruction override that confguration or should that also be altered accordingly to:

$config->timezone = 'Europe/London'; 

?

Any assistance would be appreciated.

[abdus]

1 hour ago, ridgedale said:

Just run into this problem following a reinstallation using the latest dev version 3.0.58. When I setup the testlocale.php file to check the current configuration:

<?php
    echo "<pre>Current locale:\n"
    var_dump(setlocale(LC_ALL, '0'));
    echo "\n\nAvailable locales:\n";
    echo exec('locale -a');
?>

and try to load the file in the browser I get the following error message:

You're missing a semicolon after echo statement on line 2.

1 hour ago, ridgedale said:

setlocale(LC_ALL, "en_GB.utf8");

You can provide an array of locales as well.

I would check what locales installed on the server. This http://www.binarytides.com/php-get-list-of-locales-installed-on-system/ script may help you debug it if you dont have ssh access to the server. If you do, locales -a command lists all installed locales.

[ridgedale]

Thanks for the reply, abdus.

I just managed to work out the issue and have got the file to run with the following code:

<? php;
echo '<pre>Current locale:\n';
var_dump(setlocale(LC_ALL, '0'));
echo '\n\nAvailable locales:\n';
echo exec('locale -a');

That code then returned:

Quote

Current locale:\nstring(1) "C"
\n\nAvailable locales:\nzu_ZA.utf8

I've not got multi-language support installed as it is not required. Is the locales setting referring to how the webhosting server is confiugured, the hosting account is configured or how the PW installation is configured?

When I check the available locales in cPanel I get the following list:

Quote

Arabic (العربية)       ar      
Czech (čeština)       cs      
Danish (dansk)       da      
German (Deutsch)       de      
Greek (Ελληνικά)       el      
English       en      
Spanish (español)       es      
Latin American Spanish (español latinoamericano)       es_419      
Iberian Spanish (español de España)       es_es      
Finnish (suomi)       fi      
Filipino (fil)       fil      
French (français)       fr      
Hebrew (עברית)       he      
Hungarian (magyar)       hu      
☃ cPanel Snowmen ☃ - i_cpanel_snowmen       i_cpanel_snowmen      
Indonesian (Bahasa Indonesia)       id      
Italian (italiano)       it      
Japanese (日本語)       ja      
Korean (한국어)       ko      
Malay (Bahasa Melayu)       ms      
Norwegian Bokmål (norsk bokmål)       nb      
Dutch (Nederlands)       nl      
Polish (polski)       pl      
Portuguese (português)       pt      
Brazilian Portuguese (português do Brasil)       pt_br      
Romanian (română)       ro      
Russian (русский)       ru      
Swedish (svenska)       sv      
Thai (ไทย)       th      
Turkish (Türkçe)       tr      
Ukrainian (українська)       uk      
Vietnamese (Tiếng Việt)       vi      
Chinese (中文)       zh      
Chinese (Taiwan) (中文（台湾）)       zh_tw 

This shows there is no locale currently configured for GB. This issue has not occurred before and I haven't changed the locale settings. There was no issue following the original installation using PW 2.7.x. The only thing I can think of is that because I set Europe/London as part of the Processwire 3.0.58 installation and because there is no existing English GB locale the installation process has baulked.

Any ideas on how best to proceed?

[abdus]

50 minutes ago, ridgedale said:

Current locale: /* ... */

Is this the whole output? It's been a long time since I last used CPanel, but can you pick a locale from the list and install? Have you tried it? If setlocale() can't find the provided locale, it defaults to 'C', (in your case only zu_ZA.utf8 is installed).

50 minutes ago, ridgedale said:

Is the locales setting referring to how the webhosting server is confiugured, the hosting account is configured or how the PW installation is configured?

setlocale() sets locale for the current execution context (Processwire instance) and is inherited in child scopes (unless overriden). It gets locale definitions from hosting server, but to enable a locale you may need to set something on CPanel. 

[ridgedale]

Hi abdus,

Current locale:\nstring(1) "C"
\n\nAvailable locales:\nzu_ZA.utf8

Yes, that was the whole of the output.

The long list shows the available locales that can presumably be set.

Quote

setlocale() sets locale for the current execution context (Processwire instance) and is inherited in child scopes (unless overriden). It gets locale definitions from hosting server, but to enable a locale you may need to set something on CPanel.

I've no idea an apparently South African locale is only showing as available when there doesn't even seem to be any South African locale option available under View Available Locales in cPanel. It is very odd.

Edit: I spotted the following under the cPanel Locales Documentation:

Quote

You cannot call the locale system directly from PHP applications. If you want localized text in your custom code, we strongly recommend that you use Perl or JavaScript, or that you display the text via a Template Toolkit file.

Looks like php code does ot display the locale information correctly for cPanel/WHM users.

Having downloaded the cPanel english (en) locale there doesn't appear to be anything included that should cause a problem for a UK localised installation of Processwire. 

Any ideas on how best to proceed.

[ridgedale]

Update:

Interestingly there was no warning message when I installed dev 3.0.59 locally in MAMP this morning but after I apparently successfully installed 3.0.59 in my cPanel-based hosting account the warning was displayed upon initial login to Processwire Admin again:

Quote

• Session: xxxxxx - Successful login

• Session: Warning: your server locale is undefined and may cause issues. Please add this to /site/config.php file (adjust “en_US.UTF-8” as needed): setlocale(LC_ALL,'en_US.UTF-8');

I note that both the locally installed (MAMP) Processwire config.php file and cPanel-based hosting account Processwire config.php file display identical configurations:

$config->prependTemplateFile = '_init.php';
$config->appendTemplateFile = '_main.php';
$config->useMarkupRegions = true;
$config->useFunctionsAPI = true; 
$config->dbCharset = 'utf8mb4';
$config->dbEngine = 'InnoDB';
...
$config->timezone = 'Europe/London';

If I simply add the following line of code to the top of the site/config.php file:

setlocale(LC_ALL,'en_GB.UTF-8');

will that resolve the issue?

 

[matjazp]

Try with 

setlocale(LC_ALL,'zu_ZA.utf8');