module Fluency - The complete translation enhancement suite for ProcessWire

[FireWire]

Hello community!

I want to share a new module I've been working on that I think could be a big boost for multi-language ProcessWire sites.

Fluency is available in the ProcessWire Modules Directory, via Composer, and on Github

Some background: I was looking for a way for our company website to be efficiently translated as working with human translators was pretty laborious and a lack of updating content created a divergence between languages. I, and several other devs here, have talked about translation integrations and the high quality services now available. Inspired by what is possible with ProcessWire, I built Fluency, a third-party translation service integration for ProcessWire.

With Fluency you can:

• Translate any plain textarea or text input
• Translate any TinyMCE or CKEditor (inline, or regular)
• Translate page names/URLs
• Translate in-template translation function wrapped strings
• Translate modules, both core and add-ons

Installation and usage is completely plug and play. Whether you're building a new multi-language site, need to update a site to multi-language, or simply want to stop manually translating a site and make any language a one-click deal, it could not be easier to do it. Fluency works by having you match the languages configured in ProcessWire to those offered by the third party translation service you choose. Currently Fluency works with DeepL and Google Cloud Translation.

Module Features

• Translate any multilanguage field while editing any page. Translate fields in Repeater, Repeater Matrix, Table, Fieldset Page, Image descriptions, etc.
• Translate any file that added in the ProcessWire language pages. It's possible to translate the entire ProcessWire core in ~20 minutes
• Provide intuitive translation features that your clients and end-users can actually use. Fluency is designed for real-world use by individuals of all skill levels with little to no training. Its ease-of-use helps encourage users to adopt a multilanguage workflow.
• Start for free, use for free. Translation services supported by Fluency offer generous free tiers that can support regular usage levels. Fluency is, and will always be, free and open source.
• Use more than one Translation Engine. You may configure Fluency to use either DeepL, Google Cloud Translation, or both by switching between them as desired.
• AI powered translations that rival humans. DeepL provides the highest levels of accuracy in translation of any service available. Fluency has been used in many production sites around the world and in commercial applications where accuracy matters. Deliver impressive battle-tested translation features your clients can count on.
• Disable translation for individual fields. Disable translation for multilanguage fields where values aren't candidates for translation such as phone numbers or email addresses
• Configure translation caching. Caching can be enabled globally so that the same content translated more than once anywhere in ProcessWire doesn't count against your API usage and provides lightning fast responses.
• Set globally ignored words and text. Configure Fluency to add exclusionary indicators during translation so that specific words or phrases remain untranslated. This works either for specific strings alone, or present in other content while remaining grammatically correct in translation.
• Choose how translation is handled for fields. Configure Fluency to have buttons for either "Translate from {default language}" on each tab, or "Translate To All Languages" to populate every language for a field from any language to any language you have configured.
• No language limits. Configure as few or as many languages as you need. 2, 5, 10, 20 language website? Absolutely possible. If the translation service you choose offers a language, you can use it in ProcessWire. When new languages are introduced by third parties, they're ready to use in Fluency.
• Visually see what fields and language tabs have modified content. Fluency adds an visual indication to each field language tab to indicate which has different content than when opening the edit page. This helps ensure that content updated in one language should be updated in other languages to prevent content divergence between languages.
• Render language meta tags and ISO codes. Output alt language meta tags, add the current language's ISO code to your <html lang=""> attribute to your templates that are automatically generated from accurate data from the third party translation service. Build a standards-compliant multi-language SEO ready page in seconds with no additional configuration.
• Render language select elements. - Fluency can generate an unordered list of language links to switch between languages when viewing your pages. You can also embed a <select> element with JS baked in to switch between languages when viewing your pages. Render it without JS to use your own.
• Manage feature access for users. Fluency provides a permission that can be assigned to user roles for managing who can translate content.
• Track your translation account usage. View your current API usage, API account limit, and remaining allotment to keep an eye on and manage usage. (Currently only offered by DeepL)
• Use the global translation tool. Fluency provides translation on each field according to the languages you configure in ProcessWire. Use the global translation tool to translate any content to any language.
• Use Fluency from your templates and code. All translation features, usage statistics, cache control, and language data are accessible globally from the $fluency object. Perform any operation and get data for any language programmatically wherever you need it.
• Build custom AJAX powered admin translation features for yourself. Fluency provides a full RESTful API within the ProcessWire admin to allow developers to add new features for ProcessWire applications powered by the same API that Fluency uses.
• Robust plain-language documentation that helps you get up to speed fast. Fluency is extremely easy to use but also includes extensive documentation for all features both within the admin and for the Fluency programming API via the README.md document. The module code itself is also fully annotated for use with the ProDevTools API explorer.
• Is and will always be data safe. Adding, upgrading, or removing Fluency does not modify or remove your content. ProcessWire handles your data, Fluency sticks to translating.
• Full module localization. Translate Fluency itself to any language. All buttons, messages, and UI elements for Fluency will be presented in any language you choose for the ProcessWire admin.
• Built for expansion. Fluency provides translation services as modular "Translation Engines" with a full framework codebase to make adding new translation services easier and more reliable. Contributions for new translation services are welcome.

Fluency is designed and built to provide everything you need to handle incredibly accurate translations and robust tools that make creating and managing multi-language sites a breeze. Built through research on translation plugins from around the web, it's the easiest and most friendly translation implementation for both end users and developers on any CMS/CMF anywhere. Fluency complements the built-in first class language features of ProcessWire.

Fluency continues to be improved with great suggestions from the community and real-world use in production applications. Big thanks to everyone who has helped make Fluency better. Contributions, suggestions, and bug reports welcome!

Please note that the browser plugin for Grammarly conflicts with Fluency (as it does with many web applications). To address this issue it is recommended that you disable Grammarly when using Fluency, or open the admin to edit pages in a private window where Grammarly may not be loaded. This is a long-standing issue in the larger web development community and creating a workaround may not be possible. If you have insight as to how this may be solved please visit the Github page and file a bugfix ticket.

Enhancements

Translate All Fields On A Page

Compatibility with newest rewrite of module is in progress...

An exciting companion module has been written by @robert which extends the functionality of Fluency to translate all fields on a page at once. The module has several useful features that can make Fluency even more useful and can come in handy for translating existing content more quickly. I recommend reading his comments for details on how it works and input on best practices later in this thread.

Get the module at the Github repo: https://github.com/robertweiss/ProcessTranslatePage

Requirements:

• ProcessWire  3.0+
• UIKit Admin Theme

That's Fluency in a nutshell.

The Module Is Free
This is my first real module and I want to give it back to the community as thanks. This is the best CMS I've worked with (thank you Ryan & contributors) and a great community (thank you dear reader).

DeepL Developer Accounts
In addition to paid Pro Developer accounts, DeepL now offers no-cost free accounts. All ProcessWire developers and users can use Fluency at no cost. Learn more about free and paid accounts by visiting the DeepL website. Sign up for a Developer account, get an API key, and start using Fluency.

Download

You can install Fluency by adding the module to your ProcessWire project using any of the following methods.

Method 1: Within ProcessWire using 'Add Module From Directory' and the class name Fluency

Method 2: Via Composer with composer require firewire/fluency

Method 3: Download from the Github repository and unzip the contents into /site/modules/

 

 

Feedback

File issues and feature requests here (your feedback and testing is greatly appreciated):
https://github.com/SkyLundy/Fluency/issues

 

Thank you! ¡Gracias! Ich danke Ihnen! Merci! Obrigado! Grazie! Dank u wel! Dziękuję! Спасибо! ありがとうございます！ 谢谢你

Edited August 11 by FireWire
Updated with latest module information and links

[szabesz]

Thank you for sharing it!

[jacmaes]

This looks amazing. Thanks for your hard work and generosity. 

[bartelsmedia]

Dreams come true! You should put up a donation link.

[tiefenb]

Awesome!

[Sergio]

This looks amazing!! Thank you for your hard work and generosity! Super contribution to ProcessWire!

[FireWire]

9 hours ago, bartelsmedia said:

Dreams come true! You should put up a donation link.

There's a link for that in the module config page. Was hidden in the screenshot for simplicity.

[monollonom]

For one of the latest website I did, the client asked if there could be an integration of some sort with DeepL since they were using it for translation. And now your amazing contribution is out ! It looks great and I might use it for the next project with this client next year.

Thank you !

[FireWire]

2 hours ago, monollonom said:

For one of the latest website I did, the client asked if there could be an integration of some sort with DeepL since they were using it for translation. And now your amazing contribution is out ! It looks great and I might use it for the next project with this client next year.

Thank you !

That's excellent to hear! Glad it came in time. I'm going to be launching a site that uses it in the next few weeks as it is stable and useful right now. It should have more features and time in production by the time you use it! If you get any time to play around with it or use it in the meantime I would love to hear feedback and any bugs you find. Cheers!

[FireWire]

I'm continuing work on this module as it gets more time in use and any edge case issues pop up. I'm going to be working on fast iterations while in alpha to address problems quickly so the module can get to a formal release. If you download and test/use the module please submit any issues via the Gitlab repo so it's on my radar. Be sure to check for newer versions on the master branch to get the latest stable version. The next push to master will contain a changelog so early users can track changes and decide if a module upgrade is needed. Thanks again to those who are able to help!

Here's the link to the repo: https://gitlab.com/SkyLundy/fluency-processwire

 

[bartelsmedia]

0.2.0 is now working fine. Thank you so much. One cosmetic/minor suggestion: Could you move the "translate" link from underneath the content box to above the box? It would dramatically help to minimize scrolling back and forth:

[bartelsmedia]

It is easy to translate from the default language to other languages but it doesn't seem to be possible to start with a secondary language and translate that into the primary language. Or am I wrong?

 

[FireWire]

2 minutes ago, bartelsmedia said:

It is easy to translate from the default language to other languages but it doesn't seem to be possible to start with a secondary language and translate that into the primary language. Or am I wrong?

 

Yes, that is a limitation in the on-page UI. Initial goal is to keep it simple for the user to translate from the primary language with a one click operation rather than needing a select input to choose the source language.

I prioritized the translation tool in the menu at the top is to translate content from any language to any language just in case that becomes something that is needed. That was the goal of making the link a modal so that tool can be used anywhere there has to be content translated from any language to any language without leaving the page or tab-hopping.

 

17 minutes ago, bartelsmedia said:

0.2.0 is now working fine. Thank you so much. One cosmetic/minor suggestion: Could you move the "translate" link from underneath the content box to above the box? It would dramatically help to minimize scrolling back and forth:

I had the same thought but was concerned about having it stack up against the language tabs if there were too many of them or if the screen was smaller (like in the screenshots in the original post). I could try experimenting with the translation trigger placement in the future but it would be sometime after localizing the UI so more people can use the module.

[bartelsmedia]

Can you hack into the Processwire language tabs and add a drop-down "Translate from language..."?! This would probably be the most flexible UI option.

[FireWire]

7 minutes ago, bartelsmedia said:

Can you hack into the Processwire language tabs and add a drop-down "Translate from language..."?! This would probably be the most flexible UI option.

Everything is hackable...

That would interfere with ProcessWire's existing UI where clicking on a tab that is already selected changes all of the tabs on the page to that language. I don't want the module to alter the native PW interface and take away a feature.

I may revisit the idea of translating a field to all languages at once and test it out to see if it is feasible in practice. If I can do that while being able to provide a progress bar or something then I think that would be the way to go. That would take care of the amount of work of clicking for each language, save space, reduce scrolling, etc.

[flydev]

 

Thanks you guy for this module!

If you are willing to accept pull-requests, I will send one which give ability to translate a file (eg: docx) into a CKEditor field and another one which will support my incoming field/module EditorJS.

 

And some suggestions.

About the issue #2 (formality param), as you can know which language is or not supported, you could hardcode the six unsupported languages only.

Quote

If I can do that while being able to provide a progress bar or something then I think that would be the way to go. 

Exactly what I was going to suggest (#15). We are lazy here ?

[FireWire]

17 hours ago, flydev ?? said:

 

Thanks you guy for this module!

If you are willing to accept pull-requests, I will send one which give ability to translate a file (eg: docx) into a CKEditor field and another one which will support my incoming field/module EditorJS.

 

And some suggestions.

About the issue #2 (formality param), as you can know which language is or not supported, you could hardcode the six unsupported languages only.

Exactly what I was going to suggest (#15). We are lazy here ?

Just doing it for the love of the code.

Really striving not to "lock in" hard definitions into the code like formality. A workaround I am thinking about is adding an ability for a developer to configure language/formality association with directions for them to review the API as a step to make it work. Believe it or not, I sent a request to DeepL to see if they have a way to query formality that isn't documented 

The module is essentially a DeepL API wrapper with a lot of the work being handled by the front-end JS. The module itself is going to go through some evolution before I can push it out for collaboration. I'll add a module method to work with files. I still need to firm up the structure of the data returned from the module's methods and then document it. It would be awesome to see this integrated in with your EditorJS module which looks super awesome!

I've got a separate repository for a CKEditor integration that I could use a help with. I set that up over the weekend but haven't really touched it. Just did some skeleton code to add the ability to highlight text and exclude it from translation but it's not even close to useful. PM me and let me know what you think.

[xportde]

We were on the verge of developing such a module ourselves, but you have beaten us to it ... Thank you very much for this great work!

Unfortunately we have a problem with the integration of the translation service into text fields, because our default language is set to german, as also set in Fluency module settings. But in text fields, we can only translate englisch texts from english, instead from german. Have we done something wrong?

 

Thank you in advance,

Thomas.

[FireWire]

10 hours ago, xportde said:

We were on the verge of developing such a module ourselves, but you have beaten us to it ... Thank you very much for this great work!

Unfortunately we have a problem with the integration of the translation service into text fields, because our default language is set to german, as also set in Fluency module settings. But in text fields, we can only translate englisch texts from english, instead from german. Have we done something wrong?

 

Thank you in advance,

Thomas.

Unfortunately I haven't localized the UI yet but that is the next priority item on my list. If you have your primary language set as German and the other language as English, the translation link will still say "Translate from English" but it will still translate from German to English. Right now this issue is purely aesthetic. Here's the language translating from German to English but the translate link saying the wrong text.

If you would like to change these now so you don't have to wait for a module upgrade, you can update the text strings located on the following lines in these JS files:

Fluency/src/js/fluency.js
153
160
216
473
476
490

Fluency/src/js/fluency_language_translator.js
108
168

By changing the text there the module will then fit your setup. Like I mentioned above, I'm working on having this behave correctly out of the box!

[FireWire]

Just found out from a user that the browser plugin for Grammarly conflicts with Fluency and causes errors which prevent translations from working. Grammarly is known to cause issues with some web apps and the company seems to not be interested in working on compatibility (as an example, Grammarly has ignored requests to fix conflicts with WordPress' Gutenberg interface for years). Considering how prevalent these issues are with many sites, this isn't a priority for the module's development. Just a heads up in case this becomes an issue for you or your clients. If you have any insight on this that could help solve the issue please open a ticket on the Fluency Gitlab repo.

[xportde]

Thank you for the clarification! 

 

In fact, the translations service works as expected, and the hint text is only a cosmetic issue. 

We have unfortunately discovered another problem with CKEditor fields in inline mode, where the translation is not transfered into the target field. Knowing this, we may use the regular mode for CKEditor. Maybe, it's very tricky to implement the translation service here, because the inline editor is loaded dynamically on demand.

 

Best regards,

Thomas.

[xportde]

Currently, it seems that only superuser can use the translation service.  Is it possible to enable translations by permission and role? 

[FireWire]

9 hours ago, xportde said:

Thank you for the clarification! 

 

In fact, the translations service works as expected, and the hint text is only a cosmetic issue. 

We have unfortunately discovered another problem with CKEditor fields in inline mode, where the translation is not transfered into the target field. Knowing this, we may use the regular mode for CKEditor. Maybe, it's very tricky to implement the translation service here, because the inline editor is loaded dynamically on demand.

 

Best regards,

Thomas.

Oh I have never really used CKEditor fields in anything other than regular mode. I will update the description of the module with that information and add that to the roadmap.

9 hours ago, xportde said:

Currently, it seems that only superuser can use the translation service.  Is it possible to enable translations by permission and role? 

I added a permission to the implementation at the company I work for. I will add that to this module as well. Update should be pushed soon and I'll let you know when that is available

 

 

[FireWire]

11 hours ago, xportde said:

Currently, it seems that only superuser can use the translation service.  Is it possible to enable translations by permission and role? 

This has been fixed, the translation tool can be accessed by anyone with page-edit permissions, which was the original intent of the module.

Out of curiosity- do you see a need to break down access to the translator tool to a specific permission? I'm curious about that idea but didn't think there was a use case.

New version is 0.2.2 - quick link to download the latest: https://gitlab.com/SkyLundy/fluency-processwire/-/archive/master/fluency-processwire-master.zip

[adrian]

@FireWire - just installed and on going to the Translation page, I get these errors:

PHP Warning: file_get_contents(/site/modules/Fluency/fluency_templates/el_h1.tpl.html): failed to open stream: No such file or directory in .../Fluency/classes/FluencyTools.class.php:31
PHP Warning: file_get_contents(/site/modules/Fluency/fluency_templates/el_p.tpl.html): failed to open stream: No such file or directory in .../Fluency/classes/FluencyTools.class.php:31

I installed via the URL to the zip file and the problem is that it resulted in two "Fluency" directories such that the path is actually:

/var/www/dev.grief.coach/site/modules/Fluency/Fluency/fluency_templates/el_p.tpl.html

Once I moved those files and did a modules > refresh it started working as expected.