Jump to content

Fluency - The complete translation enhancement suite for ProcessWire


FireWire

Recommended Posts

So, I've bumped into something worth considering.

When I added more languages to my client's site the existing pages inherited the untranslated URL slugs from the default language. So I've got to go to the Settings for each page and manually click the translate from English button for each language to pull a translated slug. That's burdensome, but it seemed to be working... until I had to start translating a bunch of blog tags, then some slugs wouldn't translate. I couldn't figure it out until I realized that Fluency must be pulling a DeepL translation from the English slug, not the English title. You'd think it wouldn't matter, but it does. Here's an example:

English > French:
AI App > Application IA
ai-app > ai-app

It turns out DeepL translates ai-app to ai-app in every language. I don't think it knows what ai-app means so it just returns it.

I don't know how commonly people will run into this problem, but I've been running into it a bit. Sometimes DeepL will return the slug unchanged, sometimes it will provide a slightly different translation than it did for the title. My suggestion would be for the slug translation prompt to use the default language page title when calling for a translation. I could see where pulling the title could create other surprises, such as when someone has the page title "Contact Us" but the slug "contact" they'll be getting longer slugs returned. Still I think that would be preferable to DeepL just silently failing. Maybe others feel differently. It's worth consideration though. 

  • Like 1
Link to comment
Share on other sites

I'll add, perhaps for @ryan to consider, that an equally good solution would be for ProcessWire to borrow a behavior from elsewhere. In WordPress if I want to reset a slug I just clear the field and save the post. It then auto-populates that field from the post title as it does when the post is first created. In ProcessWire clearing the slug fields and saving the page just reverts the fields to their prior state. A small, but nice feature in WordPress I'd love to see in PW. :)

  • Like 1
Link to comment
Share on other sites

1 minute ago, ryangorley said:

I'll add, perhaps for @ryan to consider, that an equally good solution would be for ProcessWire to borrow a behavior from elsewhere. In WordPress if I want to reset a slug I just clear the field and save the post. It then auto-populates that field from the post title as it does when the post is first creation. In ProcessWire clearing the slug field and saving the page just reverts the fields to their prior content. A small, but nice feature in WordPress. :)

You might find this useful: https://processwire.com/modules/page-rename-options/

  • Like 2
Link to comment
Share on other sites

Does anyone has a problem with installing Fluency (1.0.7) in Processwire 3.0.229 under PHP 8.2?

I can install it but when it comes to selecting the preffered engine I get a 503 error.

I suspect a mod_security issue since the module wants to save a serialized string (see below) to the database, when selecting the engine.

Anyone run accross this problem and knows a workaround? Maybe @FireWire can help?

Bildschirmfoto 2024-02-08 um 14.33.19 (1).png

Edited by update AG
PHP is 8.2 not 8.1
Link to comment
Share on other sites

On 2/7/2024 at 7:00 AM, ryangorley said:

I don't know how commonly people will run into this problem, but I've been running into it a bit. Sometimes DeepL will return the slug unchanged, sometimes it will provide a slightly different translation than it did for the title. My suggestion would be for the slug translation prompt to use the default language page title when calling for a translation. I could see where pulling the title could create other surprises, such as when someone has the page title "Contact Us" but the slug "contact" they'll be getting longer slugs returned. Still I think that would be preferable to DeepL just silently failing. Maybe others feel differently. It's worth consideration though. 

This is interesting and I hadn't thought of that before. I'm no expert on DeepL's internal logic, but I'd venture to guess that this "passthrough" is also what prevents it from trying to translate things like proper nouns or selections that it can't translate with confidence in accuracy. Double edged sword I guess.

Edge case- the page title could be "Take The Next Step" but the slug in English is "/contact-us/" so the translation to German would be "/de/den-nachsten-schritt-tun/" which doesn't match. It becomes more complicated when someone renames the page to "Find Out More", so the German slug is orphaned entirely. I'm not sure how prevalent that would be, but it draws a connection between page title and URL where they serve different roles.

I haven't seen this situation myself so I don't have anything to go off of or know if this is a big ask- is it possible to tweak the default language URL so that it has more parity across languages?

2 hours ago, update AG said:

I suspect a mod_security issue since the module wants to save a serialized string (see below) to the database, when selecting the engine.

This is a very interesting situation and one that I haven't heard come up or experienced myself. I am at a bit of a disadvantage because I haven't used Fluency in production since I released the rewritten module last year and that DB serialization was only introduced then. Do you have access to any logs where you could verify a mod_security issue? Is there anything in the response headers of the 503 page that could help? I can modify the the module but would like to be sure that it's not a different problem that may need to be addressed.

Link to comment
Share on other sites

1 hour ago, FireWire said:

This is a very interesting situation and one that I haven't heard come up or experienced myself. I am at a bit of a disadvantage because I haven't used Fluency in production since I released the rewritten module last year and that DB serialization was only introduced then. Do you have access to any logs where you could verify a mod_security issue? Is there anything in the response headers of the 503 page that could help? I can modify the the module but would like to be sure that it's not a different problem that may need to be addressed.

I managed to get the Fluency Module installed and the settings set correctly. I installed in on another server a copied the settings directly from the database. So, that is that. But the Module is still not working. It looks like the translation-api page cannot be accessed. I get an "Unrecognized path" error (see screenshot), when I access the page /processwire/fluency/api/translation/ directly. Since the whole thing is working on another server it must have to do with some server restrictions but I can't figure out what it could be. Any ideas?

Bildschirmfoto 2024-02-08 um 22.51.22.png

Bildschirmfoto 2024-02-08 um 22.51.40.png

Bildschirmfoto 2024-02-08 um 22.51.06.png

Link to comment
Share on other sites

7 minutes ago, update AG said:

when I access the page /processwire/fluency/api/translation/ directly

This path is only for use by AJAX to make the translation request, so accessing that URL directly won't work.

8 minutes ago, update AG said:

Any ideas?

Can you open the developer tools and go to the Network tab, try to translate something and then show me what the response body is?

Link to comment
Share on other sites

10 hours ago, FireWire said:

This path is only for use by AJAX to make the translation request, so accessing that URL directly won't work.

Can you open the developer tools and go to the Network tab, try to translate something and then show me what the response body is?

You can access it directly and get a JSON response, when the path is correct: 

{"error":"FLUENCY_METHOD_NOT_ALLOWED","message":"Fluency API Error: Method Not Allowed"}

But I didn't get anything so I looked closer to the path and saw that I had a wrong path. I had /processwire/fluency-1/api/translation/ as api-path, which was probably due that I deleted the module and reinstalled it so maybe it created this new path. I changed it now and it works.

Second: Still when I try to edit the settings page in the Module I get the 503. See the attached network response. I still think it is somehow a security issue, when trying to save to the database due to all this serialized data in the POST-request.

DB is by the way MySQL 8.0.

Bildschirmfoto 2024-02-09 um 09.51.56.png

Bildschirmfoto 2024-02-09 um 09.51.27.png

  • Thanks 1
Link to comment
Share on other sites

8 hours ago, update AG said:

I still think it is somehow a security issue, when trying to save to the database due to all this serialized data in the POST-request.

Thank you for double checking. I've seen a lot of different mod_security implementations (have done it myself) and there can be a lot of variance. I had one host change the error response to HTTP 418 I'm A Teapot (very unhelpful...) and others return other 4xx codes, but haven't seen a 503 Service Unavailable because that indicates a server failure, not an issue with the request. That's not to say that your hosting provider hasn't chosen a 5xx status and has a really strict implementation. Alas, I'll work on updating how the module stores information and push to a separate branch so you can test and see how it works in your environment.

I'm putting out a few fires today (anyone else having to deal with DMARC records?) but I'll get to that as soon as I can spare the time.

Link to comment
Share on other sites

Hey @update AG I've pushed a revision to the module that changes how configuration information is stored in the database and removed all serialization. It's on a separate branch so that you can test it. This requires that any Translation Engine related data will need to be re-configured.

Please download and test from this branch on the Fluency repository.

Link to comment
Share on other sites

On 2/11/2024 at 2:31 AM, FireWire said:

Hey @update AG I've pushed a revision to the module that changes how configuration information is stored in the database and removed all serialization. It's on a separate branch so that you can test it. This requires that any Translation Engine related data will need to be re-configured.

Please download and test from this branch on the Fluency repository.

Hey @FireWire Whatever you did, it's working now: Downloaded your branch, deinstalled the module first and than installed your new version. All is working fine. Configuration of the settings now all work without getting an error anymore. Great help. Thanks a lot.

  • Like 1
Link to comment
Share on other sites

@update AG Great to hear! I changed how Fluency stored config data. The issue you experienced helped find an edge case where the module may cause issues for some people. Your issue where you experienced two translator pages has also been fixed as well. Appreciate the feedback and helping make the module better.

I'm going to add this to the next release but will try to see if I can prevent any need to uninstall/reinstall and preserve settings when upgrading.

Link to comment
Share on other sites

  • 1 month later...

Hey all!

Fluency 1.0.8 has been released. Now available for upgrade in ProcessWire admin and in the modules directory. For the Composer fans out there, Fluency has been added to Packagist and can now be installed by running `composer require firewire/fluency`

This version contains a new feature where you can now disable translation on a per-field basis. Just check the "Disable translation for this field" checkbox under the 'Details' tab and Fluency will not be added to that field.

1168140423_Screenshotfrom2024-03-2412-39-58.png.34ed0c152b959fcd4c1977626918b24b.png

This is helpful where fields may contain different content for different languages but may not be suitable or desired for translation- email addresses, phone numbers, full string URLs, etc. It can help clean up the UI when editing pages and prevent unnecessary translations.

Also includes the following:

  • Activity overlay text has been resized and reformatted, now looks better where the size of the field is smaller.
  • Translation cache is now enabled by default as stated in the README file
  • Performs some polite cleanup when uninstalling the module.
  • Fix issue where CKEditor fields in collapsed Repeater and RepeaterMatrix fields may fail to initialize. Credit to @ryangorley for finding/reporting.
  • Fix issue where fields with combinations of languages that are and are not configured in Fluency are properly handled. Languages that aren't configured in Fluency are now indicated where others have the translation button present. Credit to @ryangorley for finding/reporting.
  • Fix for issue where the way that Fluency stored module configuration data may cause ModSecurity false positives depending on how rules were configured on that server. Credit to @update AG for finding/reporting.
  • Various minor fixes and code cleanup.

Please share any bugs here or, if possible, create an issue in the Fluency Github repo. Thanks!

 

  • Like 3
  • Thanks 2
Link to comment
Share on other sites

  • 2 months later...

Hi friends! Many thanks for this brilliant module!

It appears I've found a heavy issue concerning php 8.3.7 . In brief: Fluency crashes, giving multiple errors of this kind:

Uncaught Error: Undefined constant "Fluency\Engines\DeepL\CURLOPT_CONNECTTIMEOUT" in site/modules/Fluency/app/Engines/DeepL/DeepLEngine.php:220

Digging inside reveals the following code sample:

    $requestConfig = [
      CURLOPT_CONNECTTIMEOUT => 5,
...

CURLOPT_CONNECTTIMEOUT looks here like a constant, however it's not. PHP 8.3.4 can however handle this kind of arrays, and Fluency works like a charm. But it's not the case for PHP 8.3.7 .

I think it's worth taking this into consideration during further development. I've also created an issue on the Github module page.

Hope this will help to make this module even better 😉

Link to comment
Share on other sites

@theoreticHey there- we figured out the issue on Github but wanted to mention here for others as well.

Fluency uses CURL to make API requests to the translation service so that is required to be present on the server. I'll note this in the documentation on the next release as a requirement for the module since it's possible to use ProcessWire and never need or use CURL.

Good catch and thanks for the report!

  • Like 1
Link to comment
Share on other sites

@bernhard I did try to use it at one point. I remember that it wasn't compatible with how DeepL required it's URL parameters to be structured. I think it was the way that it had multiple GET variables with the same name and that didn't serialize properly when using WireHttp.

That may be a moot point now because the DeepL API now accepts JSON encoded POST requests so that URL issue isn't a problem.

Prior to that I ran into an issue with WireHttp not handling 404 responses in a way that I need to rely on, so it isn't generally top of mind. Long story short, if a 404 is returned, WireHttp will ignore that valid response and try again.

As an aside- I'm working with an API right now where 404 is a valid response for a record that doesn't exist, so trying 2x doubles the API credit usage with that service. May or may not be an issue in this instance, but it is something to keep in mind when HTTP requests have a hard cost associated with them.

  • Like 1
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...