Module: Languages (multi language content management)

[Soma]

Soma, so we are kind of neighbours. I’m living and working in Basel.

Hey cool, I'm from Schaffhausen and do work in Zurich. So yes, we're neighbours. Nice to meet you here on a "US" forum!

[ryan]

Based on where the majority of our traffic comes from, I think it might be considered a European forum.

[Oliver]

BTW: I saw some code parts checking on a NoMove property of templates to keep them being moved within the page tree. But I didn’t find a way to actually create a not movable template. Would be perfect to lock the language trees’ position in the hierarchy. Any idea?

[Pete]

If I'm understanding you right, you could create a template called "language_root" and set it's "Allowed template(s) for parents" status to "Home" (or whatever the template of the homepage is on your particular site). That setting is under the Family tab when editing a template anyway.

Whilst it doesn't prevent someone from reordering the pages at that level, I think that would lock those pages in place though I've not actually tested it.

[ryan]

Pete is right about that approach. Another option is that you can edit /site/config.php and change $config->advanced = true; Following that, when you edit templates, you'll see a 'system' tab that provides several other more advanced options like the 'noMove' option. These are intended primarily for PW core and module development.

[Oliver]

As I create the template as a clone of the home template on installation of the module, couldn’t I set the noMove using the templates attr() method?

At least I’ve set the parentTemplates to only accept the root page’s template.

[ryan]

That's correct. While there isn't an attr() method on the Template object, you could set it from the API like this:

$template->noMove = 1; 

or

$template->set('noMove', 1); 

(the first example routes through the set() method behind the scenes)

[Oliver]

Yeah, sorry, mixed that up. Of course "set"!

Was a bit too much dealing with InputField attributes today. Anyway, got it working. Thanks.

[ryan]

Like with jQuery, the attr() method is just way way to say "I only want to deal with HTML form attributes". Anything you set with attr() is going to end up as an HTML attribute in the form field, and likewise anything you get with it is from an HTML attribute.

Inputfields also have the usual set(), get() and direct reference ($a->b) means of access just like everywhere else in PW. You can also use them to get/set attributes just like with attr() and they are largely interchangeable… except for setting a new HTML attribute. If the attribute doesn't already exist, and you set it like $inputfield->something = '...', there's no way for ProcessWire to know that you want "something" that to be an HTML attribute. But if you set it with $inputfield->attr('something', '...'), then PW knows you intend that to be an HTML attribute.

My recommendation is to use the attr() method when setting and getting HTML attributes on an Inputfield. While it's not always necessary to do so, it does make it clear in your code when you are dealing with HTML attributes versus other properties in an Inputfield.

Btw, if you ever see reference to setAttribute() and getAttribute() with PW Inputfields, these are just verbose ways of saying: attr(key, value) and attr(key). The verbose versions are just there for internal use and hooks. So it's preferable to use attr() in most API code.

[Oliver]

The further I get, the more complex it gets. Have still to deal with some issues, before I can give you a test version covering the most important functionalities.

[Pete]

Keep up the good work - I'm looking forward to giving this one a test

[Soma]

The further I get, the more complex it gets. Have still to deal with some issues, before I can give you a test version covering the most important functionalities.

I can imagine. Do you need any support on this?

I think this will be one of the more important module for me/us.

If there would be no easy way to handle a multi language site with language SEO urls, it would be a dealbreaker.

[Oliver]

Thanks for the offer, Soma. Would help, if you tell about the functionalities you’d expect such a module to have. Maybe it gives me an hint for the one or other question I still have to get an answer for.

The real complex thing still is to keep the page mapping (associating the different language's translations of a page with each other) consistent, while allowing pages to be moved, trashed or deleted and also to providing the possibility to have pages in a language tree that aren’t sync’ed with other languages.

I implemented a sync option to automatically sync a language with the defined default language. So creation and deletion of pages in default language’s page tree will be repoduced within the synced language’s page tree all the way.

Alternatively, you can deal with the translations manually by creating them from the language tab in the page edit form (as already said: by cloning as well as by creating an empty new page).

What would you guys suggest should happen, when a page is moved outside the language’s page tree or is trashed? On sync, I guess, the associated pages have to be deleted or–in case of being trashed–also be trashed.

When a mapped page is moved within the language tree, the associated translations will be moved to the parent within their particular page tree, that is mapped to the new parent of the current languages page tree. But if you can have pages without a translation in the other language trees, what should happen on a page being moved to such unmapped page as new parent? Should this parent be replicated to the other language trees as new parent for the as well moved translations? Or should it be forbidden to move to unmapped parents?

Maybe I just should reduce the functionalities to syncing the page trees 1:1 first.

Btw: What information on the current language is needed within the template? ISO-Code? LCID? Or what else? Let me know.

[ryan]

While I've not built a multi-language site before, I kind of think that mapping sounds very convenient for creating pages.  But I wonder if you really need to do any more than that in your version 1.0? Seems like just that one feature (creating pages) would cover the biggest need that most have? Following that, people might even prefer to manage other aspects in the trees (I'm guessing). So some of the more advanced mapping stuff might be fun to save as options for a version 2.0?

The way I see it, on 80% of sites, clients will just be creating and editing pages. Whereas, the big moves, deletes and other big operations are something that more often falls in the court of the developers, and usually during a site's initial development. (just speaking in very general terms). So you might be able to cover the needs of 80% of people before adding the more complex mapping features?

[Oliver]

Ryan, just a GU thing: When the page has a template with noMove set, shouldn’t the "move" link in the page list be hidden? On my endless ways through the core files I believe I saw some code related to that. So far it doesn’t seem to work properly.

[ryan]

The "noMove" option means that the page can't have it's parent changed. But it may still be sortable with it's siblings.

The PageList uses the "move" label in a more general sense to refer to either changing the parent, sorting, or both (just depending on where you happen to drag the page to).

As a result, a "move" link may still appear in the page list even if a page's template has the "noMove" option set. In this case, the "move" link is only referring to sorting within the same parent.

If you drag the page to another parent, PW should tell you "no". Though, after getting the alert box, it might still look like it moved in the PageList, but that's only because I haven't figured out how to reverse it from the javascript (yet). The page didn't actually move. Hit reload or click "pages" and it'll be back where it was. Though let me know if you find it's not working this way. 

[Oliver]

Yeah, I’m currently reducing functionality for a first test version.

Just a general question: If page trees are synced, what page attributes/properties would you guys like to be synced? Parent, order, template, status? Anything else? Or anything definitely not?

[Pete]

If you're syncing a page so that one you've created in one language is then copied to another I would say that it's status should be unpublished by default - I'd personally want to finish translating a page in that case before I publish it.

Something that could be useful even in a version 1 would be the option to exclude copying pages with a certain template. One site I'm working on will have everything translated to different languages as the company does work in a few different places in Europe, but the news won't be relevant in different areas so I wouldn't want to sync news articles in that case. I imagine, though I obviously can't think of every scenario, that this would be a very useful feature. The only problem I can see is if you accidentally exclude a template for a page with children using a different template - what happens then? Does it just not copy the children or could it run the risk of not working at all and throwing an error? So... maybe this idea is not as simple as it first sounded

[Soma]

Yeah, I’m currently reducing functionality for a first test version.

Just a general question: If page trees are synced, what page attributes/properties would you guys like to be synced? Parent, order, template, status? Anything else? Or anything definitely not?

Hard to say really. Parent, order, status as you said I think for sure. But now I'm thinking about custom fields that's used for additional statuses, I'm thinking of the ones I need to create for each template because core doesn't have it the way I need it, like "not_in_navigation" checkbox and "not_searchable"... So if possible, there should be a way to add custom fields that needs to be synced on a template basis.

I think it will be easier to find out once a first alpha can be tested.

[Oliver]

Pete, you reached the point of the story, that has been driving me nearly crazy for over a week now.

As soon as you exclude pages, there has to be a logical way to deal

a) with its children, as they were synced

b) with synced pages maybe moved below the unsynced page as its children

Soma, that's a good point. But a complex one, I guess. Maybe a cool feature for version 2 or 3.

[ryan]

Not sure if this all applicable, but I'll mention it because I think it gets into the two issues you mentioned. You may already be doing something like this. But I'll type it out just in case any of it is worthwhile.

I do some cross-site syncing where pages from one PW site are duplicated on other PW sites and then mirrored every hour. The sites have different structures, so the mirrored pages have to remember where they came from. This is accomplished by maintaining a 'master_id' (integer) field on every page. That field contains the ID of the page at the originating site. So even if the page is moved in the structure, it continues to be mirrored since it's master_id will never change. If I want a given page to stop mirroring, then I just set it's master_id to 0.

Perhaps in your case, keeping a master_id field containing the ID of the page it was duplicated from would ensure that it could continue to be synced regardless of where it gets moved to in the future. Since synced pages would be sharing the same template as the master, they would both have a master_id field. So one way to differentiate the master from the others would be to use a signed ID… they both have the same number, but the master has a positive and the other(s) have negative number (master_id * -1). Canceling the mirroring at either end would just mean setting it to 0.

Any templates you want to mirror you would add a master_id field to in Setup > Templates. Your $pages->save() hook would monitor pages that have this field and populate it when creating a page. Likewise, they would create the corresponding page(s) (with the negative master_id) as child(ren) of any other pages that had a master_id equal to the saved page's parent_id * -1.

If you didn't want the parent to be mirroring, but you wanted its children to mirror, then the master parent would have a master_id of 0, and the other corresponding parents would have a master_id of the master parent's ID * -1. I think that the default would be that any page created that has a master_id field in it's template would be assumed to be mirrored until you set it to 0.

I may not understand the full scope of your needs, so correct me if I'm wrong… but wouldn't this sort of approach be a simple solution to the two issues you mentioned?

[Pete]

@ryan - slightly off-topic here, but can you roughly describe the scenario where you need to sync pages across multiple sites? I'm also intrigued as to how that would be acccomplished.

[Oliver]

Thanks for the input, Ryan.

Currently on installation the module creates a first language tree (english) automatically. It can be renamed to whatever language you want it to be. It is set as default language, because there always has to be one defined. The default language is the one you are redirected to when you access the root page as a visitor. Also it’s the master for all languages with sync set to 1.

I reduced the functionality for this first version to the pure syncing of the language trees to all actions performed on (mapped) pages in the default language tree, syncing templates, sort, status, parent.

I guess, in case of translations it makes more sense to keep the page hierarchy synchronous–and implement pages not synchronized by an option to exclude them from standard sync behaviour.

[Soma]

Just thinking. How would we handle images then? upload again on each page?

[ryan]

PW's page cloning supports images/files too. I suppose it just depends when the cloning takes place as to whether the images are duplicated there. You never really know if an image is going to need to be language specific or not, though it's description (alt tag) would be. I tend to think it's best to follow a path of more duplication here rather than trying to squeeze out some efficiency benefits in less duplication. Either way, any kind of image or file mirroring would seem more like a luxury than a requirement, in my opinion. If I were building a multi-language site, I would probably keep some pages for shared assets (outside of the language trees) and pull shared images from there when necessary. I'd reserve the on-page image and file fields for language specific stuff, not to be cloned across trees.

@Pete: Technically it's not different than any other web service for sharing content, except that it's a little more custom and deals with larger sets of data. This particular one is specific to a company intranet, so there aren't public examples I can link to. I won't get into the unique parts since it's not my own site, but can describe the general ideas common to most web services of this kind. The master server has a REST-based web service, which I just built with a template and assigned it to a page. That web service lets you query PW with selectors and it responds with JSON results containing all page data. The other web servers query the web service from a cron job calling PW's API from PHP command line scripts. They look for pages that have changed since the last query, and re-populate all the page's fields with the data parsed from the JSON. It mirrors all fieldtypes including files and page references, as well as child pages. It determines if something has been deleted by it's lack of presence in the JSON feed. If there's a page on one of the servers with a master_id that no longer appears in the feed, it moves the page to the trash and ultimately deletes it. This system deals with tens of thousands of pages, so it doesn't all happen in one request. Let me know if I can answer any more specific questions about it. Though maybe we should switch to another topic so we don't hijack this thread.