Multi-language page names / URLs

[ryan]

I'm pretty close to having native core support for multi-language page names. It's something I wanted to add originally in 2.1, but just didn't know exactly how without adding lots of overhead. After marinating on it for a long time, an easy way to accomplish it finally became apparent. A nice thing about it is that it does it with near zero overhead. It won't be as fancy as the LanguageLocalizedURL module, but it should be good for people that have relatively simple needs. It's the one feature that we were missing that would really let the multi-language fields sing, and it should lead the way for making more fieldtypes multi-language capable. 

It works by enabling you to specify an alternate name for each page, for each language. When a page is accessed at its alternate URL, then the language is automatically detected and set for the request. Combined with multi-language fields or multi-language alternate fields, it provides a full multi-language solution without need for multiple trees or having to use any code to set the language. It's not the right solution for all situations, but for some situations, it'll be quite nice.

Lets say you've got the page /about-us/contact/. For the "about-us" page you've set the Spanish language name to be "quienes-somos", and for the "contact" page you've set the Spanish language name to be "contacto". When the URL /quienes-somos/contacto/ is accessed, it's technically referring to the same page as /about-us/contact/, except that the user's language is automatically set to Spanish, and thus any multi-language fields output in Spanish. Calls to $page->url on any other pages also output the Spanish URLs. 

You don't have to define alternate labels for all pages if you don't want to. So long as there is just one of them in the URL (like in the rootParent, for example) then it'll be able to detect the language automatically. In order to avoid problems with having multiple URLs displaying the same content, it doesn't let you access the page with a URL like /about-us/contacto/ (English and Spanish mashup), because both of those pages have their names translated. So if you accessed such a URL, it would 301 redirect to the Spanish version. 

Here's a screenshot that might help to explain how these things are defined. This will be committed to the core within the next few days, as part of the LanguageSupport group of modules, but I'm going to leave it as an uninstalled alpha then beta module, until ProcessWire 2.4. 

[Soma]

Did I already say I love you?!

[Michael Murphy]

Thanks Ryan, this is a huge feature. Being able to do full multilingual sites, without the need for multiple trees and all from the core installation - this makes what was already a great multilingual solution, almost perfect. Looking forward to testing!

Could you elaborate on the situations where you think it would not be appropriate to use this new feature? I can't think of any.

How will this work with selectors? Can you use either language urls to get the same page? or will there be one definite url that you have to use in a selector?

I am also wondering how would this work with categories - if for example I am using a Page field for categories, and do the category filtering via the url segment, could I do for example

page-name-en/category-AA-en

and then

page-name-de/category-AA-de

to get the same result?

[Wanze]

Wow this is awesome news ryan! I will help to test for sure!

First answers from 3 swiss guys - no surprise because we have 4 languages in Switzerland and therefore lots of websites need to be multilingual

[Soma]

I'm especially eager to know how it's done and how features as published per individual language can be done (same as in our LanguageLocalizedURL module using checkboxes on each language installed?). Things like how certain future features will affect this way of setup, like versioning etc. So those thing I worried the most when multilanguage came into play, often CMS forget about them or lack features. I'm so happy to see page names are at least a shipped feature in core and thus considered "important" which I'm really thankful.

I'm really excited to try it and see test the s... out of it. Since I did part of the LLU module I know how hard it is to do it and cover most scenarios, but I'm sure you did a great job.

[ryan]

Did I already say I love you?!

Could you elaborate on the situations where you think it would not be appropriate to use this new feature? I can't think of any.

I think that technically multi-trees still provides for the most flexibility and will still be the right solution for many. There is kind of an assumption with multi-language page names that every page in the structure is applicable regardless of language. Also, I'm guessing that the LanguageLocalizedURL module will still be a good solution for many as well. But the truth is, I don't build a lot of multi-language sites, so I don't know all the scenarios as well as you guys. But I am building a multi-language site right now (that needs to support 6+ languages initially, with 17k+ pages), and the existing multi-language options weren't working quite right for this project, so decided to pursue this instead.

How will this work with selectors? Can you use either language urls to get the same page? or will there be one definite url that you have to use in a selector?

You can use either. 

I am also wondering how would this work with categories - if for example I am using a Page field for categories, and do the category filtering via the url segment, could I do for example

 

page-name-en/category-AA-en

and then

page-name-de/category-AA-de

to get the same result?

How you handle the URL segments outside of a page's actual path is up to you. As a matter of keeping it simple, I would probably stick to more abstract URL segments that aren't language specific. But your "page-name-en" and "page-name-de" as page names will set the language. 

I'm especially eager to know how it's done and how features as published per individual language can be done (same as in our LanguageLocalizedURL module using checkboxes on each language installed?).

I'll rely on you guys to tell me what I'm missing. I'll try to get this committed to the core later this week for anyone that wants to play with it. But I would hold off using it on production sites until we're further along with it. 

When the module is installed, every page in your site gets inputs for each language for the page "name" field (like in the screenshot). In order for the page to automatically switch languages based on URL, then the page itself OR one of it's parents must have the name defined in the target language. The URL is what ultimately triggers what language gets set to the user, and thus what language gets displayed in multi-language fields, static translations, etc.

Things like how certain future features will affect this way of setup, like versioning etc.

The behavior of it is basically identical to the existing $page->name field. In fact, it keeps the other language page names right alongside the 'name' field in the existing pages table. In this way, the data is searchable, selectable and loads with pages, without extra overhead. 

[pwired]

Ok that's great news. Forgive me if I'm asking something obvious here, but does this mean that I now, in a more easy way, can put little country flags on top of each page, and when a visitor clicks on one of the country flags, the text on the pages will be loaded in the selected language ? Would I link a country flag to the fields holding the text in it's language ?

[teppo]

Ok that's great news. Forgive me if I'm asking something obvious here, but does this mean that I now, in a more easy way, can put little country flags on top of each page, and when a visitor clicks on one of the country flags, the text on the pages will be loaded in the selected language ? Would I link a country flag to the fields holding the text in it's language ?

This one made me wonder whether each page will know it's own URL in each available language or just current language?

What I mean is that if you're viewing /about-us/contact/ (via that particular URL) and /about-us/ has Spanish name quienes-somos, in order to switch to Spanish version by using this method you'd want to redirect user to /quienes-somos/contact/. My question is essentially that is there a built-in way to find out that URL while viewing English version of the page?

Since you could already achieve this by traversing through parents etc. perhaps a built-in method (as nice as that would be) would also be slightly overkill..

Another thing I'm wondering is that are these names considered equal to original name field regarding overlapping page URLs etc. As an example, if you create a page with default (English) name "/suomeksi/" and then give a Finnish name "suomeksi" to your home page, is your home page going to have an URL ending with "2" or something like that by default?

Probably they are and I'm just overcomplicating things, again. Just wanted to make sure.

[Soma]

You just cycle through languages and output page url. Since it is now built in I'm sure there will be also a direct way.

Homepage has no url. But I know where you're going.

[teppo]

@Soma: I was going to add separate question about whether this will work for homepage too, but kind of thought it'd be obvious that it will. At least in my experience so far that'd seem pretty important, especially if this is going to be used as kind of a "language toggle"

[ryan]

This one made me wonder whether each page will know it's own URL in each available language or just current language?

There will be a way to get URLs for the page in any language. I haven't yet figured out where to put it, but it'll be there. Of course, you can always do this too:

$saved = $user->language; 
foreach($languages as $language) {
  $user->language = $language; 
  echo "<li>$page->url</li>";
}
$user->language = $saved; 

Probably what I'll do set set it up so that you can pass a context to $page->url, like $page->url($language); 

Another thing I'm wondering is that are these names considered equal to original name field regarding overlapping page URLs etc. As an example, if you create a page with default (English) name "/suomeksi/" and then give a Finnish name "suomeksi" to your home page, is your home page going to have an URL ending with "2" or something like that by default?

Overlapping will be okay, and I imagine necessary in some cases where a word is the same in two languages. So long as one of the parent names is language-specific, it'll still resolve to the right language. If there is a collision between the page and all parents, then at that point you would have to make something unique about the page's URL so that the pages don't have the same exact path. 

@Soma: I was going to add separate question about whether this will work for homepage too, but kind of thought it'd be obvious that it will. At least in my experience so far that'd seem pretty important, especially if this is going to be used as kind of a "language toggle"

Homepage has no path, so there's nothing to trigger a language selection there. In a setup using multi-language page names, I imagine the homepage would either be 1) a gateway asking you to select a language; or 2) be presented in the default language with links to other languages. Homepages for other languages would have to be at some other URL, like domain.com/de/ or wherever you wanted it. 

[teppo]

Homepage has no path, so there's nothing to trigger a language selection there. In a setup using multi-language page names, I imagine the homepage would either be 1) a gateway asking you to select a language; or 2) be presented in the default language with links to other languages. Homepages for other languages would have to be at some other URL, like domain.com/de/ or wherever you wanted it.

I was kind of hoping that this feature would make it possible to add alternate names for homepage too, so that if you gave a Finnish name like "suomeksi" for your homepage, "default version" would stay at "/" and Finnish version of homepage would be reachable at "/suomeksi/". That way one wouldn't need to implement any additional logic to select language at all.

I've no idea if that's even possible, though, just thought it might be useful..

[ryan]

I was kind of hoping that this feature would make it possible to add alternate names for homepage too, so that if you gave a Finnish name like "suomeksi" for your homepage, "default version" would stay at "/" and Finnish version of homepage would be reachable at "/suomeksi/". That way one wouldn't need to implement any additional logic to select language at all.

It might be possible, I will look into it. One way you could currently accomplish it is by making a child page of the homepage that serves as the rootParent. Lets say you needed to support 3 languages, with English being the default. You'd give this single page the names "en", "fi", and "de" (to match up with your language names). The template would need no fields, and the template file would contain just this: 

$homepage = $pages->get('/'); 
$language = $languages->get(trim($page->path, '/'));
if($language->id) $user->language = $language; 
echo $homepage->render();

In that manner, all of your non-default language homepages could be accessed at /fi/, /de/, etc. And you'd build your structure below that single language gateway page. You'd end up with URLs that are probably exactly what you'd want in a multi-language site, with no need to duplicate any pages. 

If you guys are interested, the LanguageSupportPageNames module is now committed to the dev branch. It's an early version only suitable for testing, so don't install on any production sites. 

[Soma]

Just did a little test round.

So far it's working great. 

Just noticed a problem when logged in and having checked another language than default for my user (in profile). When calling the url for default it stays on the user language and doesn't switch language at all. When I make my user have defualt language checked it works like a charm.

I'm using latest PW.

Edit:

Forgot about the homepage issue. This is indeed something we would need to handle. It would be great to have support for language segment in url like /de/ueber-uns/ and /en/about-us/. Yes this is how we build most sites.

Do you mean to have a root page that has the site inside? And have that gateway page names like "en" and "de" ? How would then rootParent() behave. Sure there's the workaround with parents but you get it.

Now also having maybe, if possible, some possibility to publish languages individually. How would you go for it if you need it with this support? 

However I think if there's an easy way to do it, then-or anyway the LanguageLocalizedURL becomes obsolete, because this module solves the main issues with LLU, which I'm not unhappy about. Remember it was created because there were no support in core yet.

[ryan]

Do you mean to have a root page that has the site inside? And have that gateway page names like "en" and "de" ? How would then rootParent() behave. Sure there's the workaround with parents but you get it.

With multi-language page names, the beginning "en" or "de" segment (as examples) aren't technically necessary, as the language can be determined from the page names alone. But I understand it may still be desirable, so I'll be looking for ways to handle that. But as for $page->rootParent, you'd probably want $page->parents()->eq(2) instead. Also Page::rootParent is hookable. Though I don't really want people to have to think about these things. 

Now also having maybe, if possible, some possibility to publish languages individually. How would you go for it if you need it with this support?

This could be done by adding checkbox or page reference fields to check which languages you want to be included in any given children() or find() results, etc., and then making use of it in your selectors. If that became desirable as a core feature, we'd probably implement it in some way similar, behind the scenes. 

Just noticed a problem when logged in and having checked another language than default for my user (in profile). When calling the url for default it stays on the user language and doesn't switch language at all. When I make my user have defualt language checked it works like a charm.

I will probably have to set it up to override the user's language settings when the page names module is installed. 

[ryan]

In the latest commit to the ProcessWire dev branch, the LanguageSupportPageNames module now supports setting the homepage name for any of the languages. It should work exactly like how you guys requested. 

For example, if you set the Spanish homepage name to be "es", then from that point forward, all URLs for Spanish are preceded by an "/es/". Likewise, if you access any path preceded by an "/es/" then the user's language is set to Spanish. 

If you access the homepage "/" then the default language is set. If you access the Spanish homepage at "/es/" then you get the Spanish version of the homepage.

Lets say that you set your "default" language homepage name to be "en". In this case, all default language pages become preceded with "/en/", except for the homepage itself… I wanted to avoid having two URLs that render the same exact thing.

Use of this homepage "name" is completely optional. You could choose to not use it for your default language, and then use it for other languages, or you could choose to not use it for any, or you could choose to use it for all. Basically, I think it'll accommodate just about anything you could want to do in that respect. 

Let me know if you guys get a chance to test it and if you run into any snags or other ideas. 

[ryan]

Some additional updates posted to the LanguageSupportPageNames module. Previously you could only define multi-language page names after the page already existed. Now you can define them when you are adding the page. See screenshot. 

[Soma]

Thanks Ryan, this sounds great news!

[horst]

Hi, just a pw-beginners-question:

is it possible to convert an existing site (eg 'Basic Example Site') to a Multilanguage site?

I have tried an installed core language modules, have setup languages and everything works fine. But when trying to change Fields to LanguageFields in Templates I get an error and afterwards DB is damaged:

Error:     Exception: Unknown column 'field_title.data1022' in 'field list'

[wishbone]

is it already advisable to use this new core?

I'm a bit confused about the documents on multilingual sites, older and newer, I don't know which approach is the easiest at the moment. Separate trees? multilanguage fields?

I only need a normal site in two languages (no backend languages), and if separate trees, how to manage.

Maybe LanguageLocalizedURL.module is the easiest way? (looks complicated to me, anyway...)
 

[ryan]

Error:     Exception: Unknown column 'field_title.data1022' in 'field list'

I haven't seen that particular error in a long time. What version of ProcessWire?

is it already advisable to use this new core?

The LanguageSupportPageNames module is only advised for development and or testing purposes at present. It will be ready to use in production over the next month. 

[ryan]

I just posted an update to the LanguageSupportPageNames module that appears in the 2.3 dev branch. The screenshot explains it best:

Basically, you can uncheck the "active" box for any language, and that page becomes unpublished, for that language only. It behaves the same way as an unpublished page, in that it doesn't show up in searches, and produces a 404 if you try to view it (unless it's editable, in which case you can still see it). You don't see a checkbox next to the default language, because that one is controlled the same way as before (via the page's unpublished status). 

[diogo]

It's great that multilanguage support is becoming so solid in the core. Thanks Ryan!

By the way, that seems to be a great cruise there in the screenshot, although I think the ship should continue until Konstanz (my girlfriend's family lives there )

[Soma]

Such great news! Thanks Ryan this is awesome. For the moment I have to test a little more and think about what could be added. But it' seems its all there that makes a great help to making multilanguage sites.

[Soma]

When I install the new module I get this

Error: Using $this when not in object context (line 641 of /Applications/XAMPP/xamppfiles/htdocs/pwmultilang/wire/modules/LanguageSupport/LanguageSupportPageNames.module)

PW still works but when open the module in admin I get this error. 

I use it on a new install and my local xampp has php 5.2.9