Progress on ProcessWire 2.2 and overview of multi-language support

[ryan]

I think you are on the right branch, because you wouldn't even have the LanguageSupport module if you weren't. But if you want to double check, you would go to your command prompt, and do this:

cd /wire/     (or wherever you have PW installed)
git status

The first line it outputs should say this, indicating the dev branch:

# On branch dev

While you are there, try typing this, just in case you've got a version that was missing something:

git pull

If that still doesn't do it, I'd suggest starting over with a fresh install. If after that it's still not working, I'm going to come up with some more things to try and hopefully we can determine what's up.

Thanks,

Ryan

[slkwrm]

The first line it outputs should say this, indicating the dev branch:

# On branch dev

Yes, I checked it and now I'm sure I use dev branch

While you are there, try typing this, just in case you've got a version that was missing something:

git pull

After this I've got: 10 files changed, 167 insertions,  60 delitions.

It still doesn't work, I can assume it has something to do with Linux/Windows file formats or paths differences, but it's just guesses. Now I'm going to try fresh install.

Thank you, Ryan

[apeisa]

Just tested it on my home computer (Win 7) and similar results with slkwrm. So it seems to be windows issue. I am not running xampp, just plain wamp.

[Nico Knoll]

@Ryan: I like the new "Site" link

---

I updated my post with the german translation to support the new translations.

[formmailer]

Just tested it on my home computer (Win 7) and similar results with slkwrm. So it seems to be windows issue. I am not running xampp, just plain wamp.

I have the same issue here (using xammp, both on Windows 7 and XP).

/Jasper

[ryan]

Thanks for testing. It's probably something stupid on my part like forgetting to account for the differences in Win vs. unix slashes. I'll do more research here and track down what the issue is and post an update.

Until then, I also wanted to show you how easy it is to use the translation functions in your own site templates. Here is a short video that demonstrates how to do it:

http://processwire.com/videos/processwire-language-translation-example/

[ryan]

I think that was it, the "\" vs "/" on windows vs. unix. Can't believe I didn't think about that before. I just committed an update that accounts for that, and am hoping this might fix it. Please let me know.

Thanks,

Ryan

[apeisa]

Git pull and it works!

Excellent work Ryan, thank you. I am really happy to see this working and implementation is great. Finnish is coming before Christmas!

[ryan]

Great! Glad that was it, thanks for testing it.

[apeisa]

Actually only thing that might be useful is search. But since there is so little to translate (at least so far), I don't see too much benefit from that. But in scenario where you install language and want to tune it little (maybe just one phrase or word), it can be little frustrating to go through all 20 files to find what you need to edit, instead of searching it once.

[ryan]

Nice work Antti! I'm amazed how quickly you got this done, you guys are all amazingly fast with this stuff. I must admit, Finnish is one cool looking language. I love the translation for "Access"  – it's just nice to look at ;D

Käyttäjähallinta

Actually only thing that might be useful is search.

That seems like a good idea. This one is possible, but a little more tricky than other parts of PW. This is because none of this data is in the database, it's all in JSON files on disk. Kind of like with gettext how all its data is in those .pot files or whatever they are called. Though in our case, JSON files are just text, so solutions are within reach whether based on grep or just loading the whole file and matching a string with stripos. If this is a recurring need we'll find a good way to implement it.

[ryan]

I like it, well thought implementation. Oh, and smart of you to reuse Wordpress documentation for the plurals and disambiguation by context. I must confess I didn't understand what exactly is "disambiguation by context" but it's late here in France so I'm tired and this is my excuse

I actually didn't understand it myself until I ran across the same word needing to be repeated in multiple places in the search module:

1. The "Search" placeholder text in the input[text] at the top right corner

2. The "Search" headline at the top of the page

3. The "Search" submit button in the sidebar search

So it's the same word repeated 3 times in the same file (meaning, in the same textdomain). All is well and good in English because it's the same word and who cares if it's repeated. But when it comes to other languages, text used in a headline, text used as a placeholder in an input, and text used in a button may need different translations. So this "disambiguation by context" means we can use the same word repeatedly and not have them all use the same translation. So here's what's happening in ProcessPageSearch:

$this->_x("Search", "headline"); 
$this->_x("Search", "input");
$this->_x("Search', "submit");

If some language needs those 3 to result in different translations, they can, thanks to disambiguation by context. It's a small detail, kind of like plurals, but I think its good to get the details right, even if they don't come up very often.

[apeisa]

For lazy "power users" it is easy to search files on server and find the right one. Others are just happy that there is admin UI for that

Käyttäjähallinta actually means "user management", since "access" translates little bit poorly in Finnish (it could be "pääsy" and alone it sounds just dumb).

It was very quick to translate, and actually pretty fun. Although I have already found many mistakes and clumsiness from my translation, but I don't care since I am awful with typos and it is really late It's a good start and solid translation really need more time and feedback to get details right.

But it is great (although little bit strange ) to see PW admin in Finnish!

[Nico Knoll]

In german you've got Ä, Ö and Ü you can use them normally in your translation. Encoding happens automatically I guess.

[formmailer]

In german you've got Ä, Ö and Ü you can use them normally in your translation. Encoding happens automatically I guess.

I know that you can can use them in the translation, but having å in the pagename gives some strange effects (it will be replaced with a -) and I am not sure how passwords react on these characters. (haven't tried)

Of course you have the ö ä ü and even the Eszett (ß) in German, but they are not the last character in the alphabet and aren't that important in the translation: if it doesn't work, a user can try something else. But if the instructions say (a-ö A-Ö) the Ö and ö characters need to work in that field, otherwise it would be very strange.

But obviously it would be good to know which foreign letters are allowed and which aren't. (not even speaking about Chines, Japanes, Arabic and so on....)

/Jasper

Edit: Just tested the following password: öäå1öäå1 and got following error: the password doesn't contain any letters.

[apeisa]

I just ignored (a-z) everywhere but in page name. I believe most users know their letters And yes, in Finnish we have åäö as last letters, just like in Swedish.

Passwords accept all chars but not spaces.

[formmailer]

Passwords accept all chars but not spaces.

I just checked that and that's correct but if (which is very unlikely, but still...) a user wants a password with no other letters than öäå, or maybe just 2334164Å it is not accepted, even though it is a correct password according the description.

[ryan]

Edit: Just tested the following password: öäå1öäå1 and got following error: the password doesn't contain any letters.

Currently it requires at least one ASCII letter, whether a-z or A-Z:

if(!preg_match('/[a-zA-Z]/', $value)) $this->error($this->_("Password does not contain at least one letter (a-z A-Z)."));

If it detects an error in the password, it makes it blank, requiring you to re-type it. I'm thinking I should update that preg_match to be a "\w" (any word character) rather than a-zA-Z, and that would probably solve this.

[ryan]

Slkwrm, Diogo–nice work! Just tried out both and they work great. Thank you for all of your efforts in doing these translations.

I must say, Russian is another very cool looking language to me, perhaps because it's so completely different from English (no landmarks for an English reader at least). The only time I saw Russian language as a kid was watching/reading about space programs, nuclear submarines, etc. So it brings me back to that when I see it in ProcessWire–fun and cool to see.

Is it possible to store cyrillic symbols without this transformstion to Unicode (?), so the json files are editable in any editor?

Are you saying that you want to use an encoding other than UTF-8? I think that UTF-8 is the only encoding that will let us be language neutral in ProcessWire itself. But any tool that will let you change the encoding of a text file should let you convert to and from it from the JSON files. For instance, on OS X, I use a tool called TextWrangler (free version of BBEdit) that lets me convert a file from one encoding to another pretty easily. Just be sure to convert back to UTF-8 before putting in ProcessWire. Are there any Cyrillic symbols that you aren't able to use in PW/UTF-8?

[Guillaume]

The only time I saw Russian language as a kid was watching/reading about space programs, nuclear submarines, etc.

Ahah! Watching your tutorial video, I was thinking you were a normal person, but no, nerd spotted here! 

As for Slkwrm question, I think he would like to have Unicode encoded JSON files. The actual JSON files are ANSI encoded with unicode character codes, if I understand correctly.

[ryan]

Ahah! Watching your tutorial video, I was thinking you were a normal person, but no, nerd spotted here!

You caught me!  ;D

As for Slkwrm question, I think he would like to have Unicode encoded JSON files. The actual JSON files are ANSI encoded with unicode character codes, if I understand correctly.

Just checked, and you are right it looks like the files do not show as UTF-8 when I load them in my text editor (mine show Mac OS roman encoding). I don't really know why because they are saved with nothing but PHP's file_get_contents() and the data in there is UTF-8. I'm guessing the file has no explicit encoding and it's just showing up as whatever is one's editor default (?). I suppose it doesn't really matter if one doesn't need to load the JSON file in a separate editor, but would like them to be explicit about the encoding if possible–just not yet sure how.

The only thing I can find in the file_put_contents user comments at php.net (for "utf") is this:

<?php 
$myFile = 'test.txt'; 
$myContent = 'I love PHP'; 
file_put_contents($myFile, utf8_encode($myContent)); 

But that doesn't really help us, because utf8_encode() is for converting ISO-8859-1 encoding to UTF-8, and we're dealing with text that is already UTF-8 encoded.

[ryan]

This is great, thanks Jasper!

Question for you guys: how far should we take the translations? Do you want the other parts of PW admin included in translations like:

• Setup > Fields
  
• Setup > Templates
  
• Setup > Languages
  
• Modules
  
• Access
  

Or is it better to just keep the basics translated, like the things we would have our clients use? Basically, I want to keep the scope of it small enough that people can add languages really easily. Also want some flexibility to keep adding improvements in the Setup sections without creating work for translators every time I add/update something. But maybe that's just the nature of it. There is potentially a large amount of translatable text, especially in the Setup sections (Fields/Templates). I'm just trying to determine if these parts should be included in the translations or not.

[Soma]

I'd vote for as much as possible, bring them all. ;D

But it's a valid question and should be well considered. Mainly the end user is the most important. But I think about site builders that don't speak well english, so I would go for complete, although this would add much more work and every future modules would need some translation... so definately worth considering having "system" section english. – Damn I couldn't decide.

[Guillaume]

Just checked, and you are right it looks like the files do not show as UTF-8 when I load them in my text editor (mine show Mac OS roman encoding). I don't really know why because they are saved with nothing but PHP's file_get_contents() and the data in there is UTF-8. I'm guessing the file has no explicit encoding and it's just showing up as whatever is one's editor default (?). I suppose it doesn't really matter if one doesn't need to load the JSON file in a separate editor, but would like them to be explicit about the encoding if possible–just not yet sure how.

Maybe the solution is in the json_encode method ? Check out the JSON_UNESCAPED_UNICODE parameter (http://php.net/manual/fr/json.constants.php). Example :

json_encode($string, JSON_UNESCAPED_UNICODE)

Disclaimer : Not tried myself and I'm a PHP newbie!

Edit : just discovered JSON_UNESCAPED_UNICODE need PHP 5.4.0...

[diogo]

I'm also undecided. For one side, would be nice to have it all translated, feels a bit strange to have some parts in one language and other parts in another. For the other side, I think that it's very unlikely that someone who doesn't understand english could build a site in pw... no documentation, no forums... hm... so, yep, makes sense to target the translations only for editors...

Maybe would be nice to have a hierarchy in files, so translators can choose how far they want to go. But this makes sense only if it doesn't mean work for Ryan alone