Progress on ProcessWire 2.2 and overview of multi-language support

[WillyC]

Well fu-CK I can't do sh-IT, I only speak 1 language (du-MB) and don't know cr-AP, but count me in to help test anyway. I was thinking I could make one of the less conventional translations like red-NECK, eb-ONICS, wal-MART or pr-ON so looking forward to this.

[Adam Kiss]

Well fu-CK I can't do sh-IT, I only speak 1 language (du-MB) and don't know cr-AP, but count me in to help test anyway. I was thinking I could make one of the less conventional translations like red-NECK, eb-ONICS, wal-MART or pr-ON so looking forward to this.

:D

I don't know whether this makes image of friendly community at all, but nonetheless, it made my morning

[vknt]

I can help out with the dutch translation if needed.

[ryan]

Thanks guys, this is definitely appreciated. As it is, the file translation system is largely done and ready for use. This is the part that does the same thing as GNU gettext. But that's just half of it. The other half is making multilingual 'text' fields, inputfield labels and descriptions (stuff that is in the database rather than the file system). So that's what I'm working on next, and working on it literally every day, I'm hoping to have something workable here by mid November. Since these two parts are somewhat independent of one another, I've been wondering if I should put out the file-translating language module sooner, or wait and release it all together.

[apeisa]

Separate release would be nice. That way everyone could start translating the admin already.

[ryan]

I agree, I think I will do that then. The only thing is that I feel like I need to figure out a few things with the multilingual text fieldtype/inputfield and field label/descriptions first, just in case it requires any changes to what's already been coded. I don't want to put it out there and then ask everyone to start over with anything. So I'll get a little more figured out on this end and then plan on releasing it in two parts if possible.

[apeisa]

Probably not worth if it takes more than few hours, but if it comes easily then go for it.

[ryan]

Just wanted to keep you all up to date on the multi language support being built for ProcessWire 2.2. The second half of multi language support is now functional – that means multi language support with inputfields and the basic text/textarea fieldtypes. This was necessary since the ProcessWire admin itself is a web site powered by dynamic content. So there needed to be a way to support multiple languages for any field labels and page titles. Since page titles have to be multi language, it's not much of a stretch to make other text fields multi language, so we have that too.

For every language installed on your system, ProcessWire will provide 'label' and 'description' inputs for in the fields editor. That way, when someone is editing a page, the fields they are editing will have labels and descriptions consistent with their language. You don't have to use different templates and fields for different languages.

You can designate any text field as being 'multi language' by changing it's type to be one of the multi language types. Initially this will include the following:

• FieldtypeTextLanguage (compatible with any 'Text' field)
  
• FieldtypeTextareaLanguage (compatible with any 'Textarea' field)
  
• FieldtypePageTitleLanguage (compatible with the 'Page Title' field)
  

We'll also be adding more down the road I'm sure. I think having multi language support for file/image descriptions would be especially useful, so that will probably be next after initial release.

When the user has selected another language, a call to any of those fields (like $page->title for example) will return the version in the user's language. If a value is not available, then it returns it in the default language.

Multi language fields are implemented at the database level, so when you add a language, it adds another DB field to the database tables used by any multi language fields. This enables the entire API to operate on the user's language. For instance, if the user is using Spanish as their language, a $pages->find("title=coche rápido") will locate that phrase in the Spanish field of the field_title table, rather than the English one.

While these multi language fields are pretty fun, it's good to keep in mind that having pages on your site returning different languages at the same URL is probably not a good strategy when it comes to search engines. So I still recommend structuring your site with multiple trees rather than relying on multi language fields. But I do think having the multi language fields will be handy for many things, even if indirectly. But the primary reason we have them is that they are necessary for the admin side of PW since it is itself a web application running in ProcessWire.

Here are some screenshots showing what I mentioned above. While everything is now working in the system, I still have a long way to go in working out details and optimizations. So it'll still be a few weeks before I can start sending this out, but we're on schedule and I think we've got a really good system coming together here.

[apeisa]

Amazing work Ryan! Thanks for sharing the progress. Can't wait to start testing this.

[diogo]

Great news Ryan!

So I still recommend structuring your site with multiple trees rather than relying on multi language fields

I still think these fields could be useful for this. Couldn't we have a way of calling the field on the language we want like this $page->title->es? Like this we could still prepare one page for each language, and have a friendly form for the editor.

[ryan]

I still think these fields could be useful for this. Couldn't we have a way of calling the field on the language we want like this $page->title->es? Like this we could still prepare one page for each language, and have a friendly form for the editor.

Well actually you can do this. When outputFormatting is off, your $page->title field (for example) is actually an object that contains all the language versions. If you treat it as a string, it'll return the value in the user's current language. But you can also access the other languages in the manner you indicated. Though the actual code is likely to be "es_es" rather than "es", but ultimately you can name the languages whatever you want.

When outputformatting is on (as it is by default in your templates), then $page->title is a string rather than an object. So if you wanted to grab some other language value, you'd either have to do one of the following:

<?php
// this 
$this->user->language = wire('languages')->get("es"); 
echo $page->title; 

// or this
echo $page->getUnformatted('title')->es; 

// or this
$page->setOutputFormatting(false);
echo $page->title->es; 

Like this we could still prepare one page for each language, and have a friendly form for the editor.

I might not be understanding your statement completely, but you'll be able to have a language-native form for the editor regardless of whether the fields themselves are multi-language or not. Field labels and descriptions (the labels that appear with each field in the page editor) are something separate from multi-language fields. I am actually guessing that most people won't even need multi-language fields at all. But if you look in your site tree at the 'Admin' structure, you'll see why we at least needed multi-language title fields for the PW admin. I suspect these multi-language fields will be useful to other people too, especially with web applications or anything that you login to. I've also been thinking we may be able to setup some URL magic to make it switch languages automatically without requiring different pages (like a URL that starts with /es/ but /es/ isn't actually a page in the system). That's beyond the scope of 2.2, but something we may want to strive for in making PW's multi-language support the best out there. 

[diogo]

I might not be understanding your statement completely, but you'll be able to have a language-native form for the editor regardless of whether the fields themselves are multi-language or not

By friendly, I was referring to having both languages in the same field (like on your screenshot), by opposed to how i would do it before (repeated fields for each language or even repeated pages) on a multi-language website implementation.

I was also wondering if, instead of creating different pages on the tree for each language, it would be possible to create multi-language websites by appending a parameter with the language on the last url segment (about/es), and using them like this:

$this->user->language = wire('languages')->get("$input->urlSegment(1)");

on the beginning of the template. would this change all the field calls of that page to the specified language?

[martinluff]

Great work Ryan, as ever looks like very flexible ways of working for multi-lingual sites

[ryan]

By friendly, I was referring to having both languages in the same field (like on your screenshot), by opposed to how i would do it before (repeated fields for each language or even repeated pages) on a multi-language website implementation.

It will provide multiple inputs for each language, like in that screenshot. But the thing to watch out for here is that access $page->title will output the proper version for that language. This is not desirable for SEO. So if you use these multilanguage fields, you'll want to be sure to have some language trigger in the URL itself. That way Google and other search engines can index both language versions.

I was also wondering if, instead of creating different pages on the tree for each language, it would be possible to create multi-language websites by appending a parameter with the language on the last url segment (about/es), and using them like this:

<?php
// note I updated your code example
if($input->urlSegment1) {
    $language = $languages->get($input->urlSegment1); 
    if($language->id) $user->language = $language; 
}

Yes, this will be fine to do. But maintaining the language state will be a pain. You'll have to remember to include that language urlSegment in all of your href attributes where applicable. It would be much easier to remember the language just as a $session variable, but of course that would not be SEO friendly. That's why I'd like to implement a placeholder (non-page) URL segment at the beginning that PW will know to use as a language trigger, like /es/path/to/page/. Still, that won't allow multi-language URL paths, but maybe an okay tradeoff. Ultimately a multi tree approach solves any possible issue on the front-end. And that's why I think most people will still take that approach for multi-language sites on the front-end. But at least we'll have a couple different ways that people can achieve multi language sites.

[apeisa]

Ryan, one question: will templates have labels and will they be multilang? That is actually pretty important, since editors can many times choose templates, and it would great to provide clean labels instead of computer friendly ones like "news-item" and "basic-page".

[ryan]

Yes, templates will have labels like fields do. Actually this is something I can go ahead and add before 2.2, and the 2.2 modules will just make them multi language.

[apeisa]

Great stuff. No hurry for this.

[Peter]

Found PW recently - it's great! I can do the Norwegian translation (no-NO) if needed.

[diogo]

Great Peter! I think that Norwegian was missing

[WillyC]

Offer my service to.help I do with ja-AN and share with world

[Nico Knoll]

What is ja-AN?

[MarcC]

Whaat. That's not Ainu, is it? That would be pretty awesome.

[WillyC]

ja-AN = jaPAN

for Japanese , middle letter is suppose to be left out, no? or misunderstood do i?

[MarcC]

WillyC, isn't Japanese ja-JP? The first two letters are the country code, and the second two are the actual language. Here's a slightly-related reference.

[WillyC]

shponcka! Correct.you are. It seem I have mangina today. thanking you for this. so help with ja-JP, study this soon i will.