module Fluency - The complete translation enhancement suite for ProcessWire

[ryangorley]

20 hours ago, FireWire said:

@ryangorley fix pushed to the development branch. Let me know if that doesn't solve it!

Thanks! I'll be able to test that tomorrow afternoon.

[ryangorley]

@FireWire Got delayed on my end, but I've uploaded and tested the update. It's working great. Thank you!

[ryangorley]

So, I've bumped into something worth considering.

When I added more languages to my client's site the existing pages inherited the untranslated URL slugs from the default language. So I've got to go to the Settings for each page and manually click the translate from English button for each language to pull a translated slug. That's burdensome, but it seemed to be working... until I had to start translating a bunch of blog tags, then some slugs wouldn't translate. I couldn't figure it out until I realized that Fluency must be pulling a DeepL translation from the English slug, not the English title. You'd think it wouldn't matter, but it does. Here's an example:

English > French:
AI App > Application IA
ai-app > ai-app

It turns out DeepL translates ai-app to ai-app in every language. I don't think it knows what ai-app means so it just returns it.

I don't know how commonly people will run into this problem, but I've been running into it a bit. Sometimes DeepL will return the slug unchanged, sometimes it will provide a slightly different translation than it did for the title. My suggestion would be for the slug translation prompt to use the default language page title when calling for a translation. I could see where pulling the title could create other surprises, such as when someone has the page title "Contact Us" but the slug "contact" they'll be getting longer slugs returned. Still I think that would be preferable to DeepL just silently failing. Maybe others feel differently. It's worth consideration though. 

[ryangorley]

I'll add, perhaps for @ryan to consider, that an equally good solution would be for ProcessWire to borrow a behavior from elsewhere. In WordPress if I want to reset a slug I just clear the field and save the post. It then auto-populates that field from the post title as it does when the post is first created. In ProcessWire clearing the slug fields and saving the page just reverts the fields to their prior state. A small, but nice feature in WordPress I'd love to see in PW. :)

[adrian]

1 minute ago, ryangorley said:

I'll add, perhaps for @ryan to consider, that an equally good solution would be for ProcessWire to borrow a behavior from elsewhere. In WordPress if I want to reset a slug I just clear the field and save the post. It then auto-populates that field from the post title as it does when the post is first creation. In ProcessWire clearing the slug field and saving the page just reverts the fields to their prior content. A small, but nice feature in WordPress. :)

You might find this useful: https://processwire.com/modules/page-rename-options/

[update AG]

Does anyone has a problem with installing Fluency (1.0.7) in Processwire 3.0.229 under PHP 8.2?

I can install it but when it comes to selecting the preffered engine I get a 503 error.

I suspect a mod_security issue since the module wants to save a serialized string (see below) to the database, when selecting the engine.

Anyone run accross this problem and knows a workaround? Maybe @FireWire can help?

Edited February 8 by update AG
PHP is 8.2 not 8.1

[FireWire]

On 2/7/2024 at 7:00 AM, ryangorley said:

I don't know how commonly people will run into this problem, but I've been running into it a bit. Sometimes DeepL will return the slug unchanged, sometimes it will provide a slightly different translation than it did for the title. My suggestion would be for the slug translation prompt to use the default language page title when calling for a translation. I could see where pulling the title could create other surprises, such as when someone has the page title "Contact Us" but the slug "contact" they'll be getting longer slugs returned. Still I think that would be preferable to DeepL just silently failing. Maybe others feel differently. It's worth consideration though. 

This is interesting and I hadn't thought of that before. I'm no expert on DeepL's internal logic, but I'd venture to guess that this "passthrough" is also what prevents it from trying to translate things like proper nouns or selections that it can't translate with confidence in accuracy. Double edged sword I guess.

Edge case- the page title could be "Take The Next Step" but the slug in English is "/contact-us/" so the translation to German would be "/de/den-nachsten-schritt-tun/" which doesn't match. It becomes more complicated when someone renames the page to "Find Out More", so the German slug is orphaned entirely. I'm not sure how prevalent that would be, but it draws a connection between page title and URL where they serve different roles.

I haven't seen this situation myself so I don't have anything to go off of or know if this is a big ask- is it possible to tweak the default language URL so that it has more parity across languages?

2 hours ago, update AG said:

I suspect a mod_security issue since the module wants to save a serialized string (see below) to the database, when selecting the engine.

This is a very interesting situation and one that I haven't heard come up or experienced myself. I am at a bit of a disadvantage because I haven't used Fluency in production since I released the rewritten module last year and that DB serialization was only introduced then. Do you have access to any logs where you could verify a mod_security issue? Is there anything in the response headers of the 503 page that could help? I can modify the the module but would like to be sure that it's not a different problem that may need to be addressed.

[update AG]

1 hour ago, FireWire said:

This is a very interesting situation and one that I haven't heard come up or experienced myself. I am at a bit of a disadvantage because I haven't used Fluency in production since I released the rewritten module last year and that DB serialization was only introduced then. Do you have access to any logs where you could verify a mod_security issue? Is there anything in the response headers of the 503 page that could help? I can modify the the module but would like to be sure that it's not a different problem that may need to be addressed.

I managed to get the Fluency Module installed and the settings set correctly. I installed in on another server a copied the settings directly from the database. So, that is that. But the Module is still not working. It looks like the translation-api page cannot be accessed. I get an "Unrecognized path" error (see screenshot), when I access the page /processwire/fluency/api/translation/ directly. Since the whole thing is working on another server it must have to do with some server restrictions but I can't figure out what it could be. Any ideas?

[FireWire]

7 minutes ago, update AG said:

when I access the page /processwire/fluency/api/translation/ directly

This path is only for use by AJAX to make the translation request, so accessing that URL directly won't work.

8 minutes ago, update AG said:

Any ideas?

Can you open the developer tools and go to the Network tab, try to translate something and then show me what the response body is?

[update AG]

10 hours ago, FireWire said:

This path is only for use by AJAX to make the translation request, so accessing that URL directly won't work.

Can you open the developer tools and go to the Network tab, try to translate something and then show me what the response body is?

You can access it directly and get a JSON response, when the path is correct: 

{"error":"FLUENCY_METHOD_NOT_ALLOWED","message":"Fluency API Error: Method Not Allowed"}

But I didn't get anything so I looked closer to the path and saw that I had a wrong path. I had /processwire/fluency-1/api/translation/ as api-path, which was probably due that I deleted the module and reinstalled it so maybe it created this new path. I changed it now and it works.

Second: Still when I try to edit the settings page in the Module I get the 503. See the attached network response. I still think it is somehow a security issue, when trying to save to the database due to all this serialized data in the POST-request.

DB is by the way MySQL 8.0.

[FireWire]

8 hours ago, update AG said:

I still think it is somehow a security issue, when trying to save to the database due to all this serialized data in the POST-request.

Thank you for double checking. I've seen a lot of different mod_security implementations (have done it myself) and there can be a lot of variance. I had one host change the error response to HTTP 418 I'm A Teapot (very unhelpful...) and others return other 4xx codes, but haven't seen a 503 Service Unavailable because that indicates a server failure, not an issue with the request. That's not to say that your hosting provider hasn't chosen a 5xx status and has a really strict implementation. Alas, I'll work on updating how the module stores information and push to a separate branch so you can test and see how it works in your environment.

I'm putting out a few fires today (anyone else having to deal with DMARC records?) but I'll get to that as soon as I can spare the time.

[FireWire]

Hey @update AG I've pushed a revision to the module that changes how configuration information is stored in the database and removed all serialization. It's on a separate branch so that you can test it. This requires that any Translation Engine related data will need to be re-configured.

Please download and test from this branch on the Fluency repository.

[update AG]

On 2/11/2024 at 2:31 AM, FireWire said:

Hey @update AG I've pushed a revision to the module that changes how configuration information is stored in the database and removed all serialization. It's on a separate branch so that you can test it. This requires that any Translation Engine related data will need to be re-configured.

Please download and test from this branch on the Fluency repository.

Hey @FireWire Whatever you did, it's working now: Downloaded your branch, deinstalled the module first and than installed your new version. All is working fine. Configuration of the settings now all work without getting an error anymore. Great help. Thanks a lot.

[FireWire]

@update AG Great to hear! I changed how Fluency stored config data. The issue you experienced helped find an edge case where the module may cause issues for some people. Your issue where you experienced two translator pages has also been fixed as well. Appreciate the feedback and helping make the module better.

I'm going to add this to the next release but will try to see if I can prevent any need to uninstall/reinstall and preserve settings when upgrading.

[FireWire]

Hey all!

Fluency 1.0.8 has been released. Now available for upgrade in ProcessWire admin and in the modules directory. For the Composer fans out there, Fluency has been added to Packagist and can now be installed by running `composer require firewire/fluency`

This version contains a new feature where you can now disable translation on a per-field basis. Just check the "Disable translation for this field" checkbox under the 'Details' tab and Fluency will not be added to that field.

This is helpful where fields may contain different content for different languages but may not be suitable or desired for translation- email addresses, phone numbers, full string URLs, etc. It can help clean up the UI when editing pages and prevent unnecessary translations.

Also includes the following:

• Activity overlay text has been resized and reformatted, now looks better where the size of the field is smaller.
• Translation cache is now enabled by default as stated in the README file
• Performs some polite cleanup when uninstalling the module.
• Fix issue where CKEditor fields in collapsed Repeater and RepeaterMatrix fields may fail to initialize. Credit to @ryangorley for finding/reporting.
• Fix issue where fields with combinations of languages that are and are not configured in Fluency are properly handled. Languages that aren't configured in Fluency are now indicated where others have the translation button present. Credit to @ryangorley for finding/reporting.
• Fix for issue where the way that Fluency stored module configuration data may cause ModSecurity false positives depending on how rules were configured on that server. Credit to @update AG for finding/reporting.
• Various minor fixes and code cleanup.

Please share any bugs here or, if possible, create an issue in the Fluency Github repo. Thanks!