Jump to content

Fluency - Integrated DeepL Powered Content Translation


FireWire

Recommended Posts

Hello community!

I want to share a new module I've been working on that I think could be a big boost for multi-language ProcessWire sites.

Some background, I was looking for a way for our company website to be efficiently translated as working with human translators was pretty laborious and a lack of updating content created a divergence between languages. I, and several other devs here, have talked about translation integrations and have recognized the power that DeepL has. DeepL is an AI deep learning powered service that delivers translation quality beyond any automated service available. After access to the API was opened up to the US, I built Fluency, a DeepL translation integration for ProcessWire.

Fluency brings automated translation to every multi-language field in the admin, and also provides a translation tool allowing the user to translate their text to any language without it being inside a template's field. With Fluency you can:

  • Translate any plain textarea or text input
  • Translate any CKEditor content (yes, with markup)
  • Translate page names for fully localized URLs on every page
  • Translate your in-template translation function wrapped strings
  • Translate modules

Fluency is free, and now so is DeepL
Since this module was first built DeepL has introduced free Developer accounts that allow anyone to start using Fluency at zero cost and beginning with the version 0.3.0 release Fluency now supports free DeepL accounts. As of June 2021 DeepL supports translation to 26 languages and continues to offer more.

Installation and usage is completely plug and play. Whether you're building a new multi-language site, need to update a site to multi-language, or simply want to stop manually translating a site and make any language a one-click deal, it could not be easier to do it. Fluency works by having you match the languages configured in ProcessWire to DeepL's. You can have your site translating to any or all of the languages DeepL translates to in minutes (quite literally).

Let's break out the screenshots...
When the default language tab is shown, a message is displayed to let users know that translation is available. Clicking on each tab shows a link that says "Translate from English". Clicking it shows an animated overlay with the word "Translating..." cycling through each language and a light gradient shift. Have a CKEditor field? All good. Fluency will translated it and use DeepL's ability to translate text within HTML tags. CKEditor fields can be translated as easily and accurately as text/textarea fields.

input_fields.thumb.jpg.9337edf798b103053e744fffed460b90.jpg

Repeaters and AJAX created fields also have translation enabled thanks to a JavaScript MutationObserver that searches for multi-language fields and adds translation as they're inserted into the DOM. If there's a multi-language field on the page, it will have translation added.

fluency_repeater.thumb.jpg.879cbf180c7e9714798b736e1db612f4.jpg

Same goes for image description fields. Multi-language SEO friendly images are good to go.

fluently_image_description.thumb.jpg.03269127f3d10f64a9b67d567cbdb765.jpg

Creating a new page from one of your templates? Translate your title, and also translate your page name for native language URLs. (Not available for Russian, Chinese, or Japanese languages due to URL limitations). These can be changed in the "Settings" tab for any page as well so whether you're translating new pages or existing pages, you control the URLs everywhere.

fluency_new_page.thumb.jpg.b3b649fd3087bf2c0cc565c60fed733c.jpg

Language configuration pages are no different. Translate the names of your languages and search for both Site Translation Files (including all of your modules)

fluency_language_page.thumb.jpg.63e41f268eecf2f3f3742325a3fbc5b1.jpg

Translate all of the static text in your templates as well. Notice that the placeholders are retained. DeepL is pretty good at recognizing and keeping non-translatable strings like that. If it is changed, it's easy to fix manually.

fluency_template_translation.thumb.jpg.38093371850fcb9011e9b7d668a79680.jpg

Fluency adds a "Translate" item to the CMS header. When clicked this opens up a modal with a full translation tool that lets the user translate any language to any language. No need to leave the admin if you need to translate content from a secondary language back to the default ProcessWire language. There is also a button to get the current API usage statistics. DeepL account owners can set billing limitations via character count to control costs. This may help larger sites or sites being retrofitted keep an eye on their usage. Fluency can be used by users having roles given the fluency-translate permission.

fluency_translation_tool.thumb.jpg.e69dcca7351659306731f8fffa084a20.jpg

It couldn't be easier to add Fluency to your new or existing website. Simply add your API key and you're shown what languages are currently available for translation from/to as provided by DeepL. This list and all configuration options are taken live from the API so when DeepL releases new languages you can add them to your site without any work. No module updates, just an easy configuration. Just match the language you configured in ProcessWire to the DeepL language you want it to be associated with and you're done. Fluency also allows you to create a list of words/phrases that will not be translated which can prevent items such as brands and company names from being translated when they shouldn't

fluency_config.thumb.jpg.0a0fbd7fcaefd5adda3f59ad850d708f.jpg

 

Limitations:

  • No "translate page" - Translating multiple fields can be done by clicking multiple translation links on multiple fields at once but engineering a "one click page translate" is not feasible from a user experience standpoint. The time it takes to translate one field can be a second or two, but cumulatively that may take much longer (CKEditor fields are slower than plain text fields). There may be a workaround in the future but it isn't currently on the roadmap. See a solution below...
  • No "translate site" - Same thing goes for translating an entire website at once. It would be great, but it would be a very intense process and take a very (very) long time. There may be a workaround in the future but it isn't on the roadmap.
  • No current support for Inline CKEditor fields - Handling for CKEditor on-demand hasn't been implemented yet, this is planned for a future release though and can be done. I just forgot about it because I've never really used that feature personally..
  • Alpha release - This module is in alpha. Releases should be stable and usable, but there may be edge case issues. Test the module thoroughly and please report any bugs via a Github issue on the repository or respond here.

Please note that the browser plugin for Grammarly conflicts with Fluency (as it does with many web applications). To address this issue it is recommended that you disable Grammarly when using Fluency, or open the admin to edit pages in a private window where Grammarly may not be loaded. This is a long-standing issue in the larger web development community and creating a workaround may not be possible. If you have insight as to how this may be solved please visit the Github page and file a bugfix ticket.

Enhancements

Translate All Fields On A Page
An exciting companion module has been written by @robert which extends the functionality of Fluency to translate all fields on a page at once. The module has several useful features that can make Fluency even more useful and can come in handy for translating existing content more quickly. I recommend reading his comments for details on how it works and input on best practices later in this thread.

Get the module at the Github repo: https://github.com/robertweiss/ProcessTranslatePage

Requirements:

  • ProcessWire  3.0+
  • UIKit Admin Theme

That's Fluency in a nutshell. A core effort in this module is to create it so that there is nothing DeepL related hard-coded in that would require updating it when DeepL offers new languages. I would like this to be a future-friendly module that doesn't require developer work to keep it up-to-date.

The Module Is Free
This is my first real module and I want to give it back to the community as thanks. This is the best CMS I've worked with (thank you Ryan & contributors) and a great community (thank you dear reader).

DeepL Developer Accounts
In addition to paid Pro Developer accounts, DeepL now offers no-cost free accounts. Now all ProcessWire developers and users can use Fluency at no cost. Learn more about free and paid accounts by visiting the DeepL website. Sign up for a Developer account, get an API key, and start using Fluency.

Download & Feedback
Download the latest version here
https://github.com/SkyLundy/Fluency-Translation/archive/main.zip

Github repository:
https://github.com/SkyLundy/Fluency-Translation

File issues and feature requests here (your feedback and testing is greatly appreciated):
https://github.com/SkyLundy/Fluency-Translation/issues

 

Thank you! ¡Gracias! Ich danke Ihnen! Merci! Obrigado! Grazie! Dank u wel! Dziękuję! Спасибо! ありがとうございます! 谢谢你!

fluency_results.jpg

Edited by FireWire
Updated with page translation companion module.
  • Like 28
  • Thanks 11
Link to comment
Share on other sites

For one of the latest website I did, the client asked if there could be an integration of some sort with DeepL since they were using it for translation. And now your amazing contribution is out ! It looks great and I might use it for the next project with this client next year.

Thank you !

  • Like 1
Link to comment
Share on other sites

2 hours ago, monollonom said:

For one of the latest website I did, the client asked if there could be an integration of some sort with DeepL since they were using it for translation. And now your amazing contribution is out ! It looks great and I might use it for the next project with this client next year.

Thank you !

That's excellent to hear! Glad it came in time. I'm going to be launching a site that uses it in the next few weeks as it is stable and useful right now. It should have more features and time in production by the time you use it! If you get any time to play around with it or use it in the meantime I would love to hear feedback and any bugs you find. Cheers!

  • Like 1
Link to comment
Share on other sites

I'm continuing work on this module as it gets more time in use and any edge case issues pop up. I'm going to be working on fast iterations while in alpha to address problems quickly so the module can get to a formal release. If you download and test/use the module please submit any issues via the Gitlab repo so it's on my radar. Be sure to check for newer versions on the master branch to get the latest stable version. The next push to master will contain a changelog so early users can track changes and decide if a module upgrade is needed. Thanks again to those who are able to help!

Here's the link to the repo: https://gitlab.com/SkyLundy/fluency-processwire

 

  • Like 1
Link to comment
Share on other sites

2 minutes ago, bartelsmedia said:

It is easy to translate from the default language to other languages but it doesn't seem to be possible to start with a secondary language and translate that into the primary language. Or am I wrong?

 

Yes, that is a limitation in the on-page UI. Initial goal is to keep it simple for the user to translate from the primary language with a one click operation rather than needing a select input to choose the source language.

I prioritized the translation tool in the menu at the top is to translate content from any language to any language just in case that becomes something that is needed. That was the goal of making the link a modal so that tool can be used anywhere there has to be content translated from any language to any language without leaving the page or tab-hopping.

 

17 minutes ago, bartelsmedia said:

0.2.0 is now working fine. Thank you so much. One cosmetic/minor suggestion: Could you move the "translate" link from underneath the content box to above the box? It would dramatically help to minimize scrolling back and forth:

move.thumb.png.7e2a67eed151365794a9f3a693cf8721.png

I had the same thought but was concerned about having it stack up against the language tabs if there were too many of them or if the screen was smaller (like in the screenshots in the original post). I could try experimenting with the translation trigger placement in the future but it would be sometime after localizing the UI so more people can use the module.

Link to comment
Share on other sites

7 minutes ago, bartelsmedia said:

Can you hack into the Processwire language tabs and add a drop-down "Translate from language..."?! This would probably be the most flexible UI option.

Everything is hackable...

That would interfere with ProcessWire's existing UI where clicking on a tab that is already selected changes all of the tabs on the page to that language. I don't want the module to alter the native PW interface and take away a feature.

I may revisit the idea of translating a field to all languages at once and test it out to see if it is feasible in practice. If I can do that while being able to provide a progress bar or something then I think that would be the way to go. That would take care of the amount of work of clicking for each language, save space, reduce scrolling, etc.

Link to comment
Share on other sites

giphy.gif 

Thanks you guy for this module!

If you are willing to accept pull-requests, I will send one which give ability to translate a file (eg: docx) into a CKEditor field and another one which will support my incoming field/module EditorJS.

 

And some suggestions.

About the issue #2 (formality param), as you can know which language is or not supported, you could hardcode the six unsupported languages only.

Quote

If I can do that while being able to provide a progress bar or something then I think that would be the way to go. 

Exactly what I was going to suggest (#15). We are lazy here 😂

  • Like 1
Link to comment
Share on other sites

17 hours ago, flydev 👊🏻 said:

giphy.gif 

Thanks you guy for this module!

If you are willing to accept pull-requests, I will send one which give ability to translate a file (eg: docx) into a CKEditor field and another one which will support my incoming field/module EditorJS.

 

And some suggestions.

About the issue #2 (formality param), as you can know which language is or not supported, you could hardcode the six unsupported languages only.

Exactly what I was going to suggest (#15). We are lazy here 😂

Just doing it for the love of the code.

Really striving not to "lock in" hard definitions into the code like formality. A workaround I am thinking about is adding an ability for a developer to configure language/formality association with directions for them to review the API as a step to make it work. Believe it or not, I sent a request to DeepL to see if they have a way to query formality that isn't documented 

The module is essentially a DeepL API wrapper with a lot of the work being handled by the front-end JS. The module itself is going to go through some evolution before I can push it out for collaboration. I'll add a module method to work with files. I still need to firm up the structure of the data returned from the module's methods and then document it. It would be awesome to see this integrated in with your EditorJS module which looks super awesome!

I've got a separate repository for a CKEditor integration that I could use a help with. I set that up over the weekend but haven't really touched it. Just did some skeleton code to add the ability to highlight text and exclude it from translation but it's not even close to useful. PM me and let me know what you think.

Link to comment
Share on other sites

We were on the verge of developing such a module ourselves, but you have beaten us to it ... Thank you very much for this great work!

Unfortunately we have a problem with the integration of the translation service into text fields, because our default language is set to german, as also set in Fluency module settings. But in text fields, we can only translate englisch texts from english, instead from german. Have we done something wrong?

 

Thank you in advance,

Thomas.

settings.png

text_field.png

  • Like 1
Link to comment
Share on other sites

10 hours ago, xportde said:

We were on the verge of developing such a module ourselves, but you have beaten us to it ... Thank you very much for this great work!

Unfortunately we have a problem with the integration of the translation service into text fields, because our default language is set to german, as also set in Fluency module settings. But in text fields, we can only translate englisch texts from english, instead from german. Have we done something wrong?

 

Thank you in advance,

Thomas.

settings.png

text_field.png

Unfortunately I haven't localized the UI yet but that is the next priority item on my list. If you have your primary language set as German and the other language as English, the translation link will still say "Translate from English" but it will still translate from German to English. Right now this issue is purely aesthetic. Here's the language translating from German to English but the translate link saying the wrong text.

868413029_Screenshotfrom2020-11-1109-35-00.png.311e4b7a9ca2e4558d012ab18b451e60.png

If you would like to change these now so you don't have to wait for a module upgrade, you can update the text strings located on the following lines in these JS files:

Fluency/src/js/fluency.js
153
160
216
473
476
490

Fluency/src/js/fluency_language_translator.js
108
168

By changing the text there the module will then fit your setup. Like I mentioned above, I'm working on having this behave correctly out of the box!

Link to comment
Share on other sites

Just found out from a user that the browser plugin for Grammarly conflicts with Fluency and causes errors which prevent translations from working. Grammarly is known to cause issues with some web apps and the company seems to not be interested in working on compatibility (as an example, Grammarly has ignored requests to fix conflicts with WordPress' Gutenberg interface for years). Considering how prevalent these issues are with many sites, this isn't a priority for the module's development. Just a heads up in case this becomes an issue for you or your clients. If you have any insight on this that could help solve the issue please open a ticket on the Fluency Gitlab repo.

Link to comment
Share on other sites

Thank you for the clarification! 

 

In fact, the translations service works as expected, and the hint text is only a cosmetic issue. 

We have unfortunately discovered another problem with CKEditor fields in inline mode, where the translation is not transfered into the target field. Knowing this, we may use the regular mode for CKEditor. Maybe, it's very tricky to implement the translation service here, because the inline editor is loaded dynamically on demand.

 

Best regards,

Thomas.

Link to comment
Share on other sites

9 hours ago, xportde said:

Thank you for the clarification! 

 

In fact, the translations service works as expected, and the hint text is only a cosmetic issue. 

We have unfortunately discovered another problem with CKEditor fields in inline mode, where the translation is not transfered into the target field. Knowing this, we may use the regular mode for CKEditor. Maybe, it's very tricky to implement the translation service here, because the inline editor is loaded dynamically on demand.

 

Best regards,

Thomas.

Oh I have never really used CKEditor fields in anything other than regular mode. I will update the description of the module with that information and add that to the roadmap.

9 hours ago, xportde said:

Currently, it seems that only superuser can use the translation service.  Is it possible to enable translations by permission and role? 

I added a permission to the implementation at the company I work for. I will add that to this module as well. Update should be pushed soon and I'll let you know when that is available

 

 

Link to comment
Share on other sites

11 hours ago, xportde said:

Currently, it seems that only superuser can use the translation service.  Is it possible to enable translations by permission and role? 

This has been fixed, the translation tool can be accessed by anyone with page-edit permissions, which was the original intent of the module.

Out of curiosity- do you see a need to break down access to the translator tool to a specific permission? I'm curious about that idea but didn't think there was a use case.

New version is 0.2.2 - quick link to download the latest: https://gitlab.com/SkyLundy/fluency-processwire/-/archive/master/fluency-processwire-master.zip

Link to comment
Share on other sites

@FireWire - just installed and on going to the Translation page, I get these errors:

PHP Warning: file_get_contents(/site/modules/Fluency/fluency_templates/el_h1.tpl.html): failed to open stream: No such file or directory in .../Fluency/classes/FluencyTools.class.php:31
PHP Warning: file_get_contents(/site/modules/Fluency/fluency_templates/el_p.tpl.html): failed to open stream: No such file or directory in .../Fluency/classes/FluencyTools.class.php:31

I installed via the URL to the zip file and the problem is that it resulted in two "Fluency" directories such that the path is actually:

/var/www/dev.grief.coach/site/modules/Fluency/Fluency/fluency_templates/el_p.tpl.html

Once I moved those files and did a modules > refresh it started working as expected.

  • Thanks 1
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Similar Content

    • By monollonom
      PageMjmlToHtml
      Github: https://github.com/romaincazier/PageMjmlToHtml
      Modules directory: https://processwire.com/modules/page-mjml-to-html/
      A module allowing you to write your Processwire template using MJML and get a converted HTML output using MJML API.
      This is considered to be in alpha and as such needs some testing before being used in production!

      About
      Created by Mailjet, MJML is a markup language making it a breeze to create newsletters displayed consistently across all email clients.
      Write your template using MJML combined with Processwire’s API and this module will automatically convert your code into a working newsletter thanks to their free-to-use Rest API.
      Prerequisite
      For this module to work you will need to get an API key and paste it in the module’s configuration.
      Usage
      Once your credentials are validated, select the template(s) in which you’re using the MJML syntax, save and go visualize your page(s) to see if everything’s good. You will either get error/warning messages or your email properly formatted and ready-to-go.
      From there you can copy/paste the raw generated code in an external mailing service or distribute your newsletter using ProMailer.
      Features
      The MJML output is cached to avoid repetitive API calls Not cached if there are errors/warnings Cleared if the page is saved Cleared if the template file has been modified A simple (dumb?) code viewer highlights lines with errors/warnings A button is added to quickly copy the raw code of the generated newsletter Not added if the page is rendered outside of a PageView Only visible to users with the page’s edit permission A shortcut is also added under “View” in the edit page to open the raw code in a new tab Multi-languages support
      Notes
      The code viewer is only shown to superusers. If there’s an error the page will display:
      Only its title for guests Its title and a message inviting to contact the administrator for editors If you are using the markup regions output strategy, it might be best to not append files to preserve your MJML markup before calling the MJML API. This option is available in the module’s settings.
    • By Marco Ro
      Hi guys!
      I'm a bit anxious because this is the first module I present! (beta modulo) But I will finally be able to share something with the community too! :)
      This is a BETA version of the PayPal payment system called: PayPal Commerce Platform.
      It is an advanced system (Business Pro account is needed) that brings various benefits in terms of fees and above all integrates direct payment with credit/debit cards. 
      The module integrates with Padloper 0.0.2, which is the current installation I'm using.
      This system integrates the classic PayPal buy button, the alternative or local payment method and the new payment system: credit/debit cards that doesn't go through the PayPal account. It is a Stripe-style payment, it connects directly with the bank and integrates 3D security validation.
      I say that it is a BETA because this module currently only works with Sandbox account, to put it live you need to change API url manually (manually for the moment).
      Because this module is not ready for live:
      I would like to have your opinion on how I built the module (is the first one I do). I don't want to share something that is not fish but I need a comparison with someone more experienced than me, for be sure that this is the best way to code the module.
      If you want to try this I created a git, you will find all the instructions for installation and correct operation. (Git has a MIT licensed)
      https://github.com/MarcooRo/processwire-PayPal-Commerce-Platform I hope I did something that you guys can like :)
    • By monollonom
      (once again I was surprised to see a work of mine pop up in the newsletter, this time without even listing the module on PW modules website 😅. Thx @teppo !)
      FieldtypeQRCode
      Github: https://github.com/romaincazier/FieldtypeQRCode
      Modules directory: https://processwire.com/modules/fieldtype-qrcode/
      A simple fieldtype generating a QR Code from the public URL of the page, and more.
      Using the PHP library QR Code Generator by Kazuhiko Arase.

      Options
      In the field’s Details tab you can change between .gif or .svg formats. If you select .svg you will have the option to directly output the markup instead of a base64 image. SVG is the default.
      You can also change what is used to generate the QR code and even have several sources. The accepted sources (separated by a comma) are: httpUrl, editUrl, or the name of any text/URL/file/image field.
      If LanguageSupport is installed the compatible sources (httpUrl, text field, ...) will return as many QR codes as there are languages. Note however that when outputting on the front-end, only the languages visible to the user will be generated.
      Formatting
      Unformatted value
      When using $page->getUnformatted("qrcode_field") it returns an array with the following structure:
      [ [ "label" => string, // label used in the admin "qr" => string, // the qrcode image "source" => string, // the source, as defined in the configuration "text" => string // and the text used to generate the qrcode ], ... ] Formatted value
      The formatted value is an <img>/<svg> (or several right next to each other). There is no other markup.
      Should you need the same markup as in the admin you could use:
      $field = $fields->get("qrcode_field"); $field->type->markupValue($page, $field, $page->getUnformatted("qrcode_field")); But it’s a bit cumbersome, plus you need to import the FieldtypeQRCode's css/js. Best is to make your own markup using the unformatted value.
      Static QR code generator
      You can call FieldtypeQRCode::generateQRCode to generate any QR code you want. Its arguments are:
      string $text bool $svg Generate the QR code as svg instead of gif ? (default=true) bool $markup If svg, output its markup instead of a base64 ? (default=false) Hooks
      Please have a look at the source code for more details about the hookable functions.
      Examples
      $wire->addHookAfter("FieldtypeQRCode::getQRText", function($event) { $page = $event->arguments("page"); $event->return = $page->title; // or could be: $event->return = "Your custom text"; }) $wire->addHookAfter("FieldtypeQRCode::generateQRCodes", function($event) { $qrcodes = $event->return; // keep everything except the QR codes generated from editUrl foreach($qrcodes as $key => &$qrcode) { if($qrcode["source"] === "editUrl") { unset($qrcodes[$key]); } } unset($qrcode); $event->return = $qrcodes; })
    • By Sebi
      AppApiFile adds the /file endpoint to the AppApi routes definition. Makes it possible to query files via the api. 
      This module relies on the base module AppApi, which must be installed before AppApiFile can do its work.
      Features
      You can access all files that are uploaded at any ProcessWire page. Call api/file/route/in/pagetree?file=test.jpg to access a page via its route in the page tree. Alternatively you can call api/file/4242?file=test.jpg (e.g.,) to access a page by its id. The module will make sure that the page is accessible by the active user.
      The GET-param "file" defines the basename of the file which you want to get.
      The following GET-params (optional) can be used to manipulate an image:
      width height maxwidth maxheight cropX cropY Use GET-Param format=base64 to receive the file in base64 format.
    • By MarkE
      This fieldtype and inputfield bundle was built for storing measurement values within a field, rendering them in a variety of formats and converting them to other units or otherwise modifying them via the API.
      The API consists of a number of predefined functions, some of which include...
      render() for rendering the measurement object, valueAs() for converting the value to another unit value, convertTo() for converting the whole measurement object to different units, and add() and subtract() for for modifying the stored value by the value (converted as required) in another measurement. In the admin the inputfield includes a checkbox (which can be optionally disabled) for converting values on page save. For an example if a value was typed in as centimeters, the unit was changed to metres, and the page saved with this checkbox selected, said value would be automatically converted so that e.g. 170 cm becomes 1.7 m.

      A simple length field using Fieldtype Measurement and Inputfield Measurement.
      Combination units (e.g. feet and inches) are also supported.
      Please note that this module is 'proof of concept' at the moment - there are limited units available and quite a lot of code tidying to do. More units will be added shortly.
      See the GitHub at https://github.com/MetaTunes/FieldtypeMeasurement for full details and updates.
×
×
  • Create New...