Centralized Translation of Processwire and PW-Documentations

[lisandi]

This Thread is opened new as it will only discuss a centralized solution! nothing more and nothing less! The quotes are coming from the former Thread which was actualy about finding Language files on Mac and had nothing to do with the actual problem of PW!

In Processwire there are 2 major problems with translations when working especially with Multidomain sites in one processwire DB

1. The default can not be defined on a per site base like in other CMS with Multidomain support in one DB.

i.e. the default language get's defined by the guest user but this guest user is existing only once! You can't define a guest user on a per Domain Base in a multidomain installation which is working with one DB. Solution: Instead of defining the default language by the language of the guest user it woudl be advised to have some code or dropdown or whatshowever to choose the right default language on a per site base. This can happen i.e. through code inserted in a specific site template belonging to one domain.

2. Working with translations in Processwire is a hazzle - really and it could be much much easier. We are just testing it if Weblate or Pootle could do the job and the same with the documentation to probably use readthedocs / sphinx instead.

To reduce workload on the developer site and also to increase the quality of documentation and translations it is very important to be able to collaborate with others on this. Therefor a local solution (in a module or elsewhere locally) can never be the right way in terms of smoothness, economie and workflow.

As we need at least 2 - 3 translations in nearly every site we are working with including the translation of documentation (especially those for editors and site admins) we will work on a solution which lateron might help to solve the problems of all. I only hope that the modules don't get cluttered up with lots of localized files now. - If this would habben than each module we are using here will have soon also translations of Thai, Khmer, Laos, Burmese, etc in it and for sure nobody in Europe would need them! Keeping things centralized on a centralized system is the much much better way for translations and documentations.
 

Not sure I understand your logic. Why each of the churches need to translate all files themselves?

This is the case if you need to run all churches from their own DB as some have german, some English, some Thai, some even Arabic as default language! And for sure we won't handle such an amount maintaining all those translations - that's why the churches would need to handle their stuff all by them selves which will end up in a chaos and finally in a no go command from those churches to use Processwire! Most of those Churches work with volunteers and are not those Mega Million Churches with Cathedrales etc. like you see often in the US. They have usually a dedicated webteam, but ours are very tiny units and need to keep costs at anabsolut minimum.

The same applies to the multidomain School Servers with many school/classroom websites in them etc. 90% of them would need another default language than English and the default languages are changing depending on the location where they are.
 

No need for any third party service, but updating translations through modules directory would be nice.

Well as pointed out above it will be a real hazzle to keep the quality of translations and the standards if there is no centralized solution in doing all the translating stuff. Updating a translation can happen by a module and should happen by a module - no question! - But instead of storing all translations inside a module it would be adviced to keep them separated and store only one language (the default language of the module - which could also be German or even Thai) inside the module. The module in the backend would than check with the translation server which new translations would be available and download those tranalations ONLY for those modules you have installed. Those translations than get storred in a separate directory where wll those translations can be managed or even be overwritten partly by a second local translation folder for only local translations. With a fall back methode it gets checked which translation is available and should be used.
 

is the translation available in the folder for local translation yes no?
If no than check in the global translation folder yes no?
If no download it from the language server?

yes always means = use it.

And the update process would be similar but simply skipping the local translation folder. This way you can ensure that modules always have accurate and updated translations.

Finally you might run that process than via a cronjob or manually - up to you!

---

Translations itself will be done on Weblate or Pootle and docs (so our idea on readthedocs or a global sphinx)

Module editor provides the default language (which could be any - but the language would be needed to be specified by a code - i.e. en_UK, de_CH, de_DE etc.

A parser will parse the modules directory for those default language file and convert the json into mo/po for use in Pootle. With Weblate we could integrate json directly
On Weblate/Pootle those files can be translated in a collaborative effort, even by none developers, but i.e. by people who are good in translating stuff - this keeps the quality high and it will reduce the workload on developer sites as they don't need to worry about the translations or even don't have to integrate them into their modules etc.

Now an integrator sets up a website and defines the default and other languages. A module will now check if there are new translations available for the installed languages and they can be downloaded and stored in the site like described already above.

--

Maintenance Tasks would be quite low on the site of the Weblate/Pootle Server site. The benefit of Weblate would be that they have a FREE hosted Version of webslate of OpenSource Projects - so no maintenance stuff would on PW site concerning the translation server which is great. (but we would need to change some stuff - see below)

Readthedocs is very well maintained already and it is completely free! An ideal way to do documentations and meanwhile used by lots of projects and parts of projects. And of course the module which will manage the translations needs to be maintained to so that it will continue working when the core gets updated. But this would be the case anyway with any module ;-). But here we would need to manage the connection to weblate/pootle. But also here the hosted webslate solution would be the best way to go as it can parse github repos and the translations of readthedocs will be actually hosted on a github ;-) In other words it will simply work.

using Pootle it woudl be a little bit more work as the Pootle server would need to be maintained as pointed out already.

--
Extendibility of that idea:
1. Integrating a backend part to translate / propose translations from within PW

2. Doing the translation of strings with a local application (which than will store those localized translations in the folder for local translations, like described above.) This can already be done by a module and this module only would need to be adjusted (enhanced) to work with translatins coming from the central langauge repository.

--

WEBLATE - READTHEDOCS - GITHUB
We discussed it here in our Team and we think that it is the best way to go! If there are any other ideas let us know - but for sure we are not interested in a solution which will clutter up module and corefiles with all kind of translations/documentations which we won't use on our sites! And it must be a solution which allows collaboration with others to reduce workload.

Readthedocs can work with transiflex but until now we can't see a better free solution than weblate or pootle
http://www.oneskyapp.com/comparison/transifex-alternative/

Some links for those interested:
Readthedocs (maintained and hosted)
https://readthedocs.org/

Weblate (maintained and hosted)
http://weblate.org
https://hosted.weblate.org/
http://weblate.org/en/features/
http://weblate.org/en/hosting/

Pootle (selfhosted)
http://pootle.translatehouse.org/

We propose the following way to go IMHO
https://hosted.weblate.org/ + https://readthedocs.org/ + https://github.com/

Benefits:
We could reuse the JSON
Collaboration on translations would be available
Documentation would be available for translation to
The code gets already maintained on github while I suggest to open up another repo in which all modules (also those not approved meanwhile by the ryan team) could be integrated for translation and documentation. This would help to get translations and documentations done already in an earlier state than stable.

If you know a better maintained and free solution like we proposed above to be used for a collaboartive translation and documentation effort, please point it out and describe the benefits of it in comparison to what we propose here.

---

Important REMARK:

This thread is NOT meant to discuss if there should be a centralized or only a local solution! Please keep all stuff concerning this out of that Thread and discuss it elsewhere. This Thread here is ONLY! about a globalized collaborative solution where the translations get managed at a centralized place and where documentation gets written on a centraliced place and where we create a module to integrate all those translations and documentations.

Who would be interested to work with us on getting things done and to move the modules and documentation to weblate and readthedocs. PM us! Thanks.

 

[Beluga]

No need for any third party service, but updating translations through modules directory would be nice.

I disagree with this. It doesn't even have to be 3rd party, but can be self-hosted like pointed out above. I've used Transifex and Crowdin and they make things smooth. Developers can just pull everything at once whenever they wish.

Another self-hosted solution is http://www.mediawiki.org/wiki/Extension:Translate and this for easy deployment http://www.mediawiki.org/wiki/MediaWiki_Language_Extension_Bundle

Existing file format support: http://www.mediawiki.org/wiki/Help:Extension:Translate/File_format_support

[Ivan Gretsky]

I think it is a very important topic lisandi brings up. Lack of decent translation for PW (at least for languages other than Deutsch and Finnish) is the thing that could limit its propagation. There is a Russian translation module, but it is quite outdated. I still can't figure out how to easily update it - the translation workflow is complex and not documented in any way. There are probably no translations for any modules in Russian.

This topic was brought up several times by many but was never made clear. Managing translations could require some adjustments to the software itself. As our community has a distinguished leader and most architectural decisions are made with some sort of his approval I would ask Ryan to step in. We are eager to solve our problems ourselves, but we want to do it together in a non-contradictory way. Let's decide on something ))

Translations are a step to make PW more wide spread and therefore to make its development and support more economically advantageous, as I see it.

[lisandi]

Hi Beluga

Your idea is also a possible way. We actually use it at TYPO3 to document errors and write some  additional documentation but we think our suggested solution would be the better choice.

Often in threads about translations or translation servers it was talked about the amount of time needed to maintain and fix the system itself. This will be always the case with selfhosted solutions. Therefore we are currently following the idea to pull the core and modules on a separate github account where it would be easier to collect also all those modules which are not available in the processwire repository right now. Translations could actually already start in a muchearlier stage than a stable release IMHO.

This will be connected to weblate and readthedocs. This way we will be able to write documentation on readthedocs and to do the translation part of the modules on weblate. Weblate is supposed to be able to handle json files, which are actually monolingual files.

As nobody from the Processwire community had to worry about the weblate or readthe doc or github server we would only need to get all the docs and files to be translated to weblate. If we have figured out the best way in doing that this process could be automated by a script we think on a daily or even half daily base.
 

We are eager to solve our problems ourselves, but we want to do it together in a non-contradictory way. Let's decide on something ))

well said Ivan, you are absolutely right!

Thanks

Andi
 

[jtherczeg]

When choosing the most appropriate localization platform, before obliging with Weblate, please consider if it supports different access levels for translators. Regular translators would contribute actually to the rough translation, while proofreaders (having more translation experiences, may also have permission to translate) can check the quality of the provided translations. When a localization is in quality assurance stage, regular translators may only suggest translations, when they find a better solution.

Another important criteria if the API of the chosen localization platform supported the automatic file conversion to the required JSON format. I recently translated for Sourcefabric on Transifex. After a certain interval of time the affected files were automatically updated in their GitHub repository.

As a result of this intention for centralization, a further step may be to connect to this translation repo during the installation process of ProcessWire and select and download the needed languages.

Regarding the documentation, why an external solution is needed? The Translate extension for MediaWiki enables you to translate the content into different languages. Install MediaWiki on a subdomain and start to work.

[Beluga]

Hi Beluga

Your idea is also a possible way. We actually use it at TYPO3 to document errors and write some  additional documentation but we think our suggested solution would be the better choice.

Yeah, I was just telling about my experience, I have no preference except that it should be centralized. I've only used non-self-hosted solutions thus far and I've had no problems. I just looked at the Weblate blog and was pleased to find it has been under very active development. So I say +1 for hosted.weblate.org.

Anyone curious can try their demo server, user: demo pass: demo

@jtherczeg: Weblate seems to have very fine-grained access control:

http://docs.weblate.org/en/latest/admin/auth.html#privileges

I guess this is true in hosted.weblate.org as well?

[lisandi]

Hi Jtherczeg

Transifex

Transiflex is not really FreeOpenSourceSoftware (Free = Freedom) and it unfortunately also not Free (Free = like Free Beer). Readthedocs woudl be connected already to Transiflex by the way but I think with weblate we can do the same as they themselve use Readthedocs for their own documentation.

please consider if it supports different access levels for translators.

supported the automatic file conversion to the required JSON format.

Weblate does support translation of monolingual resources like JSON. http://docs.weblate.org/en/weblate-1.7/admin.html (scrol down also for finegrained access rights - it is really great with weblate!
"You enter Git repository location and file mask which files to translate and Weblate automatically fetches the Git and finds all matching translatable files."
The initial problem is actually only to get all language files into the right order - into a logical one into the repository.

Right now we choose the structure to have a translation for the core in one gitbranch

and the modules in another - we need to test out what is best way doing it! As we aourself ned to use also modules which are NOT in the priocesswire repository and those which are simply in a non stable state but used already, we will try to collect all modules to that repository and update it regularly via scripts which git pull from the original repos. The problem here is that with a git pull also the translations would be overwritten. So much better would be actually to have a centralized repositopry also for development where all modules could be than continously be updated. Another problem we already figured out is that the processwire repository seem s not to be maintained in terms of looking for modules which get no more maintained at all by their module owners/creators. This is causing a buit a chaos of modules which than even won't update anymore correctly. We have here one site where wesimply install all those modules in and than when one get updated in the processwire repository the great modulemanager module will alarm us that an update is available. Doing the update than with thos no more maintained modules results sometimes than in incorrect updates or incorrect notifications (versions) that the update actually took already place. We contacted some of the module owners and some are fizing it and others are simply no more with Processwire ;-). This might cause also problems for the translations but we will see!

Regarding the documentation, why an external solution is needed? The Translate extension for MediaWiki enables you to translate the content into different languages. Install MediaWiki on a subdomain and start to work

You are right mediawiki could be used immediately when it woudl get installed on a server.

Which server - the one of processwire.org
Who is maintaining the server

Who will have access with what rights

Who is taking care for the wiki itself

...
and I guess some more questions which are more or less obsolete by using readthedocs ;-)
Beside this each individual could create his own readthedocs account for their custom documentation and than pull that custom documentation they write for THEIR individual customers into multiple site of them.

The problem with Mediawiki or also with Pootle is that 1st this server needs to be maintained by someone and MediaWiki and it wuld be the same if using Pootle itself needs to be maintained by someone beside the fakt that we would need someone managing theaccess stuff which would also be needed on any other solution. Wit Readthedocs we would have no problems at all with their Server (Cloud), or with the Software Readthedocs is using. The maintenance Footprint is very very small by using Readthedocs. And the docs look really good! Check it out.
 

Media Wiki and Readthedocs would be by the way both external solutions as the documentation will take place outside of processwire. Than the needed docs - and only those needed by the modules/core and by the installed translations should get pulled into a Processwire installation and updated regularly like we update modules already in processwire. Beside that there should be the possibility to modify locally an translation which than gets not overwritten anymore by the updates of a modules-language file.

Here by the way the way Macura is writing the docs inside of Processwire could be quite helpful. we only would need to pull the original translation i.e. via RSS and than i.e. could overwrite them in another tab on a page. Each module which gets installed would also install their documentation page in the admin area, which would than incllude this feed and the local Tab - with the customized translation. The tabs itself for each module than would need to be able to be enabled (default) or disabled i.e. when there is a complete customized version or both enbled when thereis simply additional information a dev is writing for a customized site.

Another great benefit of weblate would be that everyone can actually install a local version of it to for himself. It is real Free Software.
The same way you can install readthedocs locally and you can of course have also your own git - in other words you can collaborate butit woudl offer you all choices actually to also not collaborate totaly or in parts without loosing the ability to pull docs from the centralized solutions.

Of course the maintenance effort would be than by each one self who is maintinainng his own local servers with weblate, readthedocs and/or git ;-)
 

[Ivan Gretsky]

I do not see any obstacles to test out weblate + github for translations. I searched weblate repo and did not find Processwire there yet. Lisandi, why don't you start the project and configure it as you are most familar with the software. If it can't be done the way PW stores translation strings please report back so we can deal with it.

If the translation project grows and becomes important for the community, you could then hand off or share the administration rights with those interested (and probably Ryan first).

We can deal with documentation and integration later. What do you think about it?

[lisandi]

Great idea!

This is the place we intend to collect all modules. We got already the core in - ok we need to manage the updates later on a regular base.
https://github.com/pw4all

Have a look to
https://github.com/PW4ALL/ProcessWire/tree/master/site-languages/install/files

Here the translations for German and Finnish got stored in 1012 and 1013 while the German folder contains lots of translated modules already and the Finnish one only 2 translations actually.

The way those translations get stored and the folder name is actually not really a good way doing it.

Better would be to name the German Folder de_DE, or de_CH, or de_AT as depending on the region you are there are differences in the translations and would need to be reflected in a site translation or the one of the modules.

The same actually applies to en_US, en_CA, en_UK, en_AU etc which are all not the same even most words probably are the same or similar.

Another source of actual problems is the original text as this does not get stored in a json file!

let's talk about an example:

go here:
https://github.com/PW4ALL/ProcessWire/blob/master/site-languages/templates/_main.php

you can see the original text parts as they look like:

No 1 - echo _x('en', 'HTML language code')

No 2 - __('Edit')    

No 3 - echo _x('Search', 'placeholder')
No 4 - echo _x('Search', 'button')

No 5 - echo __('Powered by ProcessWire CMS')

No 6 - sprintf(__('Logout (%s)')

No 7 - __('Admin Login')

and compare it with the German translation:
https://github.com/PW4ALL/ProcessWire/blob/master/site-languages/install/files/1012/site--templates--_main-php.jso

{
"file": "site\/templates\/_main.php",
"textdomain": "site--templates--_main-php",
"translations": {
"6f9fc985da145fcdd180f254dd161c57": {
"text": "Suche"                                  --- echo _x('Search', 'placeholder') -> No 3
},
"2c889b7622e6ed4e56a1414ac333c814": {
"text": "Suche"                                  --- echo _x('Search', 'button') -> No 4
},
"7dce122004969d56ae2e0245cb754d35": {
"text": "Bearbeiten"                             --- __('Edit') -> No 2
},
"3ee693d376f73e2bfa34e985c30bec66": {
"text": "Abmelden (%s)"                          --- sprintf(__('Logout (%s)') -> No 5
},
"b6e31daf2404aab3d78c7e972dfe3f8d": {
"text": "Login"                                  --- __('Admin Login') -> No 6
},
"7679c0bdbd87c1984bd611c24b3cd0cb": {
"text": "de"                                     --- echo _x('en', 'HTML language code') -> in Head Section No 1
}
}
}

No 7 is missing in the German translation!

and with the one of the Finish translation:
 

{
"file": "site\/templates\/_main.php",
"textdomain": "site--templates--_main-php",
"translations": {
"7dce122004969d56ae2e0245cb754d35": {
"text": "Muokkaa"                           --- No 2
},
"91e2c8e591043b9d9c7611a185358068": {
"text": "Toimii ProcessWire CMS"            --- No 7
},
"3ee693d376f73e2bfa34e985c30bec66": {
"text": "Kirjaudu ulos (%s)"                --- No 5
},
"b6e31daf2404aab3d78c7e972dfe3f8d": {
"text": "Kirjaudu"                          --- No 6
},
"6f9fc985da145fcdd180f254dd161c57": {
"text": "Haku"                              --- No 3
},
"2c889b7622e6ed4e56a1414ac333c814": {
"text": "Haku"                              --- No 4
},
"7679c0bdbd87c1984bd611c24b3cd0cb": {
"text": "fi"                                --- No 1
}
}
}

as you can very easy see there is abolute no logical order in those translation files.

Similar to PO files the translations are assigned to a certain string. Without an external tool actually or an integrated solution translation is a hazzle like this.

only as a comparison to that here a file from WordPress, where you can see a very clear structure and order but also here you need to translate stuff in an external program which creates the right matches so it will work with your templates.
 

#: E:\!our theme wordpress\residence\!versions\1.07/wpresidence/404.php:16
msgid "Page not found"
msgstr ""

#: E:\!our theme wordpress\residence\!versions\1.07/wpresidence/404.php:20
msgid ""
"We're sorry. Your page could not be found, But you can check our latest "
"listings & articles"
msgstr ""

#: E:\!our theme wordpress\residence\!versions\1.07/wpresidence/404.php:24
msgid "Latest Listings"
msgstr ""

#: E:\!our theme wordpress\residence\!versions\1.07/wpresidence/404.php:44
msgid "Latest Articles"
msgstr ""

#: E:\!our theme
#: wordpress\residence\!versions\1.07/wpresidence/advanced_search_results.php:452
#: wordpress\residence\!versions\1.07/wpresidence/templates/search_unit.php:19
msgid "Search Parameters: "
msgstr ""

#: E:\!our theme
#: wordpress\residence\!versions\1.07/wpresidence/advanced_search_results.php:458
msgid "Save this Search?"
msgstr ""

#: E:\!our theme
#: wordpress\residence\!versions\1.07/wpresidence/advanced_search_results.php:459
msgid "Search name"
msgstr ""

#: E:\!our theme
#: wordpress\residence\!versions\1.07/wpresidence/advanced_search_results.php:460
msgid "Save Search"
msgstr ""

#: E:\!our theme
#: wordpress\residence\!versions\1.07/wpresidence/advanced_search_results.php:472
msgid ""
"Login to save search and and you will receive an email notification when new "
"properties matching your search will be published."
msgstr ""

#: E:\!our theme
#: wordpress\residence\!versions\1.07/wpresidence/advanced_search_results.php:506
#: wordpress\residence\!versions\1.07/wpresidence/search.php:32
msgid ""
"We didn't find any results. Please try again with different search "
"parameters. "
msgstr ""

Wordpress always needs an aditional mo file!
 

Þ•$,8˜9Project-Id-Version: wpresidence
POT-Creation-Date: 2014-11-17 10:56+0200
PO-Revision-Date: 2014-11-17 10:56+0200
Last-Translator:
Language-Team:
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Generator: Poedit 1.5.4
X-Poedit-KeywordsList: _;gettext;gettext_noop;_e;__
X-Poedit-Basepath: .
X-Poedit-SearchPath-0: E:\!our theme wordpress\residence\!versions\1.07

and TYPO3: you can very easily edit the xml file in any editor if you like!
 

<?xml version="1.0" encoding="UTF-8"?>
<T3locallangExt>
  <data type="array">
    <languageKey index="de" type="array">
      <label index="mlang_labels_tablabel">Informationen über Module</label>
      <label index="mlang_labels_tabdescr">Zeigt diese Seite über die vorhandenen Module an.</label>
      <label index="mlang_tabs_tab">Über Module</label>
    </languageKey>
  </data>
</T3locallangExt>

TYPO3 translations always hava ow also the xliff file

<?xml version='1.0' encoding='utf-8'?>
<xliff version="1.0">
  <file source-language="en" datatype="plaintext" original="messages" date="2011-10-17T20:22:32Z" product-name="aboutmodules" target-language="de">
    <header/>
    <body>
      <trans-unit id="mlang_labels_tablabel" xml:space="preserve" approved="yes">
        <source>Information about modules</source>
      <target state="translated">Informationen über Module</target></trans-unit>
      <trans-unit id="mlang_labels_tabdescr" xml:space="preserve" approved="yes">
        <source>Shows this page about available modules.</source>
      <target state="translated">Zeigt diese Seite über die vorhandenen Module an.</target></trans-unit>
      <trans-unit id="mlang_tabs_tab" xml:space="preserve" approved="yes">
        <source>About Modules</source>
      <target state="translated">Über Module</target></trans-unit>
    </body>
  </file>
</xliff>

Go and compare also the folder structres and where those translations get stored and you will realize that it is

very easy in TYPO3 - typo3conf/l10m/de/modulename/mod/de_locallan_mod.xml
especially also because each module has the original translation already in an xml file available (same format!) They work with placeholder texts!

a bit more complicated to find in Wordpress - wp-content/themes/themename/languages/themname.po - allways depending on a theme

and actually quite complicated in Processwire as even the location changes during the installation process from site/install/files to site/assets/files
site/assets/files/1012/site-templates--_main-php.json

and than even in another language the files have the same name!
site/assets/files/1013/site-templates--_main-php.json

without having a look inside one of those files you even don't know what language they are!

----

while you have in WordPress and in TYPO3 the original language or the placeholder name ALWAYS available in a readable form it makes it way easier to translate.

====================

My suggestion rightnow would be to ask Ryan for a clear statement on how the translation is actually done in TYPO3 so that we can figure out a feasible way to get all the originals into the translation system. Right now this system is actually ONLY working if people stay brave in English in their originals which limits Processwire already and lowers probably even it's quality as people are more or less forced to use probably a language they can only translate via Google and to be honest - Google is nice for public usage but not at all suitable for a professional translation.

Questions to people who did the translation module of processwire and Ryan.

1. What would be the best way to pull out the translations in the original file? What format as the original is stored in php files and not in json files unfortunately
2. How can those location strings be generated and assigned?

3. Would there be a chance to change the way / file formats, translations get done. Probably we don't even need a big change as IMHO only a Json file with the original Language would be needed ;-)

Benefit of this idea:

Processwire would NOT depend on any default language and could set any language as default and define even a very flexible fallback of languages.
In other words.

All original languages which have been already been inserted into processwire .php files would beed to get extracted to a json file which is in English language.
As there are different flavours of languages which are quite important when selling websites, the numbering need to be changed and I suggest to use de_DE, en_US, de_CH, de_AT, en_UK etc instead.

Many English, German or any other Language flavours would be possible after that change.

After the complete extraction of those language translation terms inside the defaul PHP files to jSON files those default PHP files could also hold also simple placeholders instead of the original translations. This would make it for non English spoken Developers much easier to contribute. They would set a "marker" and than add their localisaztion which could be german, russian and thai etc and the english one would simply be missing until somebody comes and is able to translate it into english language.

If a separate JSON file in english would be available than also the problem with setting a default language would be solved very easily.

I think there should be a way to parse all .php files for

__

_x
_n

and extract all those terms into one external json file per .php file.

It would use a standardised labeling - i.e. en_US-locallang-modulename.json

in another step we can collect all those en_US-locallang-....json files and put them in a centralized folder i.e. files!

I personally would avoid using files for that purpose as keeping all translations in a translation folder or simply call it "languages" would be much much easier.

As than those files are in another location and also there woudl be now an original english language file which itself could be translated into several english flavours, only the paths to those newly created languages and localisationfiles would need to be changed.

the structure at the end would look like this

site/languages/default/default-locallang-module1name.json

site/languages/default/default-locallang-module2name.json

site/languages/default/default-locallang-module3name.json

...

site/languages/en_US/en_US-locallang-module1name.json

site/languages/en_US/en_US-locallang-module2name.json

site/languages/en_US/en_US-locallang-module3name.json

...

site/languages/en_GB/en_GB-locallang-module1name.json

site/languages/en_GB/en_GB-locallang-module2name.json

site/languages/en_GB/en_GB-locallang-module3name.json

...

site/languages/de_DE/de_DE-locallang-module1name.json

site/languages/de_DE/de_DE-locallang-module2name.json

site/languages/de_DE/de_DE-locallang-module3name.json
...

site/languages/de_CH/de_CH-locallang-module1name.json

site/languages/de_CH/de_CH-locallang-module2name.json

site/languages/de_CH/de_CH-locallang-module3name.json

...

now it would be very easy to pull ONLY the files from site/assets/languages/*/*.json into weblate and translate them. As Json files are monolingual we woudl need to define a default and this could be the default folder (even it would be a duplicate of any other language, but this folder would simply hold all "placeholder" strings which could be in English or German or simple placeholders.

This site/languages folder could regularly check with the core and all modules i.e. on pw4all github if there are any new strings which would need to be translated. It would update the default files but all language files woudl stay the same. Those strings which are i.e. missing in default now would be marked as to be deleted in the translation files (but carefull as perhaps some older modules from people who don't update their sites would need them) and newly added translations would be marked as new and to be translated. This way a quite fast workflow could be created to get translations up and running very fast as a community based effort using weblate.

site/docs/default/default_doc_module1name
site/docs/default/default_doc_module2name
site/docs/default/default_doc_module3name
..

site/docs/en_US/en_US_doc_module1name
site/docs/enUS/en_US_doc_module2name
site/docs/en_US/en_US_doc_module3name

...

Concerning the documentation I suggest to do actually exactly the same like with the languages and use readthedocs for the default and present alo the translations on readthedocs. also this documentation gets stored on github in a documentation folder which simply lists the modules.

Each read the docs file could than be pulled if needed in a site from there.

-----

But the first step would be to get Ryan involved here as if the core won't provide a base json file it is a bit crappy to translate the rest! - there really needs to bea better logical order and structure in folder/ file-namings and inside the translation .json file itself.

----

So the first step is done but I need to wait now until those files will appear.

I took a site we use for our purposes and it has nearly all modules already translated into german language ;-)

That's fine.

I created on a new repository in where we will store all those translation files.
I already changed the folder namings to be de_DE, fi_FI like it should be.

Next I will try to ecxxtract all the English stuff as there is no JSON file available right now but irt would be very usefull.
I will let you know as soon as we get further on weblate and than see how to continue.

If you can see a way how to extract all the english defaults into one JSON file this would be very appreciated!

We now continue also setting up the docs on readthedocs

People who want to join this effort please contact us via PM so we can start creating translation and documentation teams.

---

You are right Ivan that it is a lot to read but IMHO it is worth it and we need people who like to contribute so they need to know the idea behind it.

Concerning Ryan you are also right that he is probably busy with other stuff and that is why we already started right the way. But I hope that he can at least get his thoughts on how to extract the default translations and how to probably change the page numbering 1012 to a de_DE etc. I guess that this won't be a big deal. If you have ideas please contribute them here as I am sure people will read and follow the thread.

[Ivan Gretsky]

Lisandi! I am sure there are some better ways to manage translations than this one used in PW now. But I think it is not wise to expect Ryan to change this system at once. You write some clever stuff, but not sure that many have enough patience to get into the topic and read so many letters . I suggest we do try to get the best out of what we have by some small enhancements at first, and plan for complete rewrite in PW 4.0.

As I see it languages are pages in PW now, and translation files are stored under assets/files/#### where #### is the number of the page (can vary from installation to installation for the same languages). Each file there represents the translation for one core file where the translatable strings can be found. Those files are in JSON and do not contain the original string which is always in default language (English).

We need to export the default language files with all the strings translated to github repo, import them to Weblate, do the translation, export to github repos corresponding to languages and then (for now) manually install them in manually created language pages. I am stuck on step one - "export the default language files with all the strings translated". Can get all the files without translations, but not with the default string values.

[lisandi]

As I wrote I took the German Translation as it seems quite good and complete (most translated)

You can find it here:
https://github.com/PW4ALL/ProcessWire-Weblate/tree/master/site/languages

as there are no English JSON files we woudl need to create them like a normal translation like I wrote above.

Concerning using a PageID or using a PageName this can be a very little change IMHO. Instead of the ID we would choose the pagetitle ;-) and this would be than de_DE instead pf PageID 1012 and fi_FI instead of PageID 1013 in my Example I described above.

For older sites there could be even a fallback like: Choose the Page Title if available, otherwise choose the PageID

To keep things much clearer as they are right now I moved all translations into the first "site" level = site/languages/de_DE/.....json file
Like in WordPress and in TYPO3 wie would use a separate language folder. (also this change for the langue files is actually only a path which would need to be changed in Processwire - so it is not a big deal. Also here could be a simple question in the next releases asking what folders should be used: site/assets/files/1013/.....json or site/languages/de_DE/.....json.

and as you wrote already in Version 4.0 or perhaps already in 3.0 this could be changed to site/languages/de_DE/....json as default and people still would be able to do itthe old way by checking the fallback (old) solution. Usually if things work fine and things get translated on the server many more people will start using and finally use only the translations from weblate. As we don't change the naming of the json file itself actually all those translations could be simply been pulled from github for everybody and than inserted in a very traditional way too without any further changes. ;-)

To update all those translation files and also to aff all those translation files which are currently already available I will extend our transaltion processwire site so that we can install here all available modules. We can use it also to see which nes get updated and which not. A cronjob will than regularly update this site and also copy the language.json files.

If you have an idea what would be best to assure that the already translated translations get not overwritten by updating the modules peobably with anolder JSON file, please let me know.

Scenario:
we install a module in the site and have the site translated into another language, which means there is a translaotion.json file probably available for that module already.
Some time later we update this module and again the translation probably gets updated (and is probably overwriting the old one grrr). IMHO we should move a module translation only once to the translation server and everything later has to be pulled from the translation server! After we have moved actually all cor and available module files to the weblate this will be a one stop solution as people will simply use it as it will provide much better translations for much less effort as it is community driven ;-)

But OK we simply need to start it to get everything up to weblate as fast as possible. I will look in that next week and upload and update all I can get on translations in German. I use German as it has already a great community like many other Open Source CMS and I am German and can understand it ;-) In parallel you could start looking after the russian stuff too and add more translations.

As I pointed out already ru_RU would not change the filenames in that folder and therefore could be easily uploaded to the site. We could even modify the easyinstalllanguage Module in pulling the languages from the pw4all/ProcessWire-Weblate repository.
 

[lisandi]

I am just loading all available translations and realized that there is also a naming inconsitency which should be standardized throughout Processwire.

1. two letter code for language (small letters) underscore two letter code flavour (capital letters)

Right now the easy language install mdule shows on the add page the correct way in fdoing it but than on the installed languages both parts are in small letters only which is not consistent and confusing!

Also German exists twice as de and as de_DE and finnish is only fi too!

On https://github.com/PW4ALL/ProcessWire-Weblate/tree/master/site/languages it is all now standardized. the languages are right now just uploading. I assume that this is right now only the translations of the core.
 

Name Title Core Translation Files Site Translation Files default English 0 0 de German 119 2 fi Finnish 0 2 zh-cn Chinese 84 0 cs-cz Czech 99 0 de-de German 122 0 es-es Spanish 25 0 fr-fr French 67 0 vi-vn Vietnamese 70 0 he-he Hebrew 36 0 it-it Italian 38 0 ja-jp Japanese Language Pack 70 0 hu-hu Hungarian 135 0 nl-nl Dutch 24 0 pl-pl Polish 37 0 pt-pt Portuguese 20 0 re-gr Processwire-greek-language 20 0 ru-ru Russian 68 0 sk-sk Slovak 37 0 se-se Swedish 23 0 tr-tr Turkish 135 0

also here the naming is quite inconsitent http://modules.processwire.com/categories/language-pack/ and should be corrected (actually those would no more be needed) if weblate will be used for future centralized translations. Or better would be to simply update them than from the translation git. Spanish exists even doubled with same name! es_ES which is not usable and confusing! Turkish has exactly the same problem and is existing twice. Collaboration would avoid it and marking a flavour with a different 2 letter Flavo code woudl help to clarify what s actually needed and what not!

If there are more translation files of the core please let me know so I will add them to https://github.com/PW4ALL/ProcessWire-Weblate/tree/master/site/languages.

Thanks

[Ivan Gretsky]

This is what I think we should ask to be done - generate the "translation" files for the default (English) language. We surely should base all further work on that and should be able to update the base translation easilly by parsing the core with in-build tool and meging the results to github.

I plead the gurus of PW forum to help us with that.

[lisandi]

So the en_US language folder shows 134 core and 256 module files to be translated.

I have uploaded also all modules and templates and created a "modules crashing sites" folder where I have taken those modules whch crashed the site during installation or at any later point. It is mainly the templating stuff like TWIG and SPEX afaiks.

I am still waiting for the confimration but actually everyone can already help to translate those files.
I will see if I can create a Profile with all those modules for transtaion purposes. Let's see. Than everybuddy could help to get things done! using a local installation and than pushing the stuff to github again.

or another way would be to use our translation installation (it has the multilanguage package frontend and backend.. But here I would need to see if it is possible tor restrict users to edit only their language and don't touch others. Perhaps sopmeone knows how to do that in Processwire.

This way we would not need at first place weblate and could start immediately until they confirm but the downside would be that there won't be a review process. But I think if we have teams they can coordinate themselves and check by themselves as it will be mainly them afterwards who will need those translations.

Right now all processwire modules which can be installed by the modulemanager have been installed and could be translated currently I am running core 2.5.10

All translations still get stored in a traditional way in assets/files on a page with PageID but I will than move those to github into the site/languages/ folderwith the correct two letter codes for language and flavour.

From there everyone can pull the translations again to his site and store it where he ants ;-) until we have a better solution which first checks the files folder for a local translation and if there isn't any available than checks the languages folder and use that one.

If your module is not on that github send me the link and I will install and than upload it if it installs without any problems on 2.5.10 + (always the highest version) Please keep your modules updated and working with the upcoming versions.

It also would be good if the Japanese and greek Translators would shorten their package name to simply Japanese and Greek like all the other ones did already. Thanks.

The files are still loading so give it some time until it will appear on git. I will let you know.