FireWire

Okay now I got it. That is the URL that needs to be in the JS file I mentioned. In that JS it does it's best to parse the URL, get the admin slug, then build the request URL. If you revisit that code in fluency_tools.js set the value of requestUrl to where it should be able to send requests to the module. It should look like this: var moduleRequest = function(requestData, callback) { var winLoc = window.location, params = [], requestUrl = 'https://domain.com/_tmp/processwire/fluency/data/' // requestUrl = winLoc.protocol + '//' + // winLoc.host + '/' + // winLoc.pathname.split('/')[1] + '/fluency/data/' // ... ommitted for brevity ... } If you move to production and ProcessWire is being served from the root directory then change this back.

Go to the Fluency module configuration page, then click on the "Translation" link in the menu bar. What is the URL of the page that loads?

Added a fix for the PayPalm img. Thanks for mentioning. Will be included in the next release. I hadn't anticipated an installation in a subfolder. The module creates an admin page when installed and that page's URL is an endpoint for the JavaScript on the page to make AJAX calls to when getting/sending translation data. The JS assumes that this URL will be at the root level. The JavaScript can't tell if the site is in a subdirectory. I don't know how to code the JavaScript to figure that out since it is separate from the CMS and the module which can tell what the full proper URL is but the JS doesn't have a way to get that info. There is probably a way to solve this problem but it will take a little investigating. There is a quick and dirty solution that would fix this issue for you right now. That would be to go into the module directory, navigate to /src/js/ and then edit fluency_tools.js to point to the full URL you need by changing line 32. From this: winLoc.pathname.split('/')[1] + '/fluency/data/' To this: winLoc.pathname.split('/')[1] + '/_temp/fluency/data/' That will let the JS know where to get/send translation data. If/when you need to move your PW site to the root directory then change it back to the original value. Let me know if that works for you.

ProcessWire definitely shines in the security department and the login page does have throttling. I don't know what your threat model is but usually sites have to deal more with bots than individual bad actors, but that depends on the purpose/use for your site. I've built many PW sites over the years and not one has ever been compromised. In my opinion, your biggest threat as far as hackers go would be a person that came into possession of one of the users' login credentials or much more common automated attack scripts with other vectors like requests containing data that the code isn't prepared to handle, or MySQL injections in your forms. Here are some additional thoughts on your login URL. If you don't want bad actors to know the login URL then it should be changed it to one that is practically unguessable like /clientname-admin-login or something easy to remember but hard to guess. Consider that your /processwire login URL is not unknowable because it's the default. Someone intent on getting to the login page or knowing what CMS you are using could analyze the markup and see that your images are being served from /site/assets/files/{page id} and any assets like JS may be served from the templates directory like /site/assets/templates/scripts- the file structure consistent across ProcessWire sites. When you load a page on a ProcessWire site, the server returns the request with the header "x-powered-by: ProcessWire CMS" unless you've disabled it and that will let someone look up what the default URL would be. Using /processwire in JS isn't really the issue. All that aside, it's not automatically a disadvantage to have your login URL known. Most of the PW sites that are built probably keep /processwire. My take is that if you have the opportunity to change it and reduce the chance non-authorized people would see the login page to begin with, why not? If this were my project I would change the login URL and create a dedicated /api page in the admin then use a hook to hide it in the page tree on render for non-superusers. It would keep the main site tree tidy as desired, allow for a clean semantic URL to work from, and not hard code in references to a part of the website that isn't content-related. If you want to take security against bots into consideration I would look into adding some bot blocking rules to your .htaccess file, I use 7G Firewall. Bot blocking prevents bot HTTP requests from requiring ProcessWire to boot up, analyze the URL, determine that it doesn't exist, and then serve the 404 page- this improves performance by reducing server load. Blocking bots also takes care of many other automated malicious requests as described above. I kinda blew this open a bit, but I think your question about security being limited to hackers and the login page could benefit from a wider scope.

I had initially written off mass translation because after seeing the time it took to translate one or two fields I thought it could be a big holdup for the user. Have you run into any situations where a whole page runs into problems? This is really awesome work!

@Robin S Appreciate the help! I was also thinking about a possible conflict between the selector and the value. Will work with your ideas. Cheers!

Hey all. I have a search feature built into our website but it won't match a whole word containing a character, in my situation the % character. When I look for "50%" it brings up results with 50 in it, but does not match pages with 50%. I've tried with several selectors including *=, **=, ~=, and attempted an advanced search with #=+50%, but no dice. I've also tried with and without sanitizers including $sanitizer->selectorValue(); Any insights? Thanks!

I'm all about devdocs.io We need to boost those Github stars...

It would be pretty complex to create settings for that in the module config on a per-field basis and I think it might be too much of an edge case to implement in the module directly so I think hooking would be your best bet on this. Using `addHookAfter();` is the way to go. Just remember to test the output of `___executeData()` as it returns different types of data depending on the request, see each case in the switch statement in the `___executeData()` method. Here's some quick info that might help you get up to speed more quickly. All of the return values from `___executeData()` have the same data structure for predictability when used. At the very least each return value has `data` and `httpStatus` properties/values. A good way to make sure you are dealing with a translation is to decode the JSON (we'll say it's assigned to $decodedResponse), and then look for `$decodedResponse->data->translations;`. If the `translations` property exists in `data` then you know it's a translation. You would want to append your string to `$decodedResponse->data->translations[0]->text;`, encode `$decodedResponse` again, then return the modified JSON.. I'm not sure how you would be able to detect what field has been translated though because the return data from `___executeData()` doesn't contain any field name so your hook would append to all multi-language fields. Let me know if you can't find a workaround for this and I'll see if I can help. Dunno if that is what you were looking for but hope it helps!

New version released. Fixed issue when translating larger volumes of text. In testing I was successful in translating over 20,000+ words in one field. When you get to large volumes of content like that there is a very noticeable amount of time it takes to complete that request, but it does it. Download the newest alpha here: Fluency 0.3.2 This should solve your issues @B3ta, let me know if you experience any issues.

@B3ta That was all totally wrong. I was improperly making the DeepL API call with the wrong method which was totally my fault. After working on the issue I've tested translations with a lot of content and have had success up to ~22,000 words (rich text in a CKEditor field). If your PW fields has more than that, I'm impressed haha. That will have to be the new ceiling and I'll engineer a solution if people start hitting that. When working with content up to those levels though I think DeepL may be being generous with their stated API request size limit so keep in mind that number could fluctuate- just hypothesizing though. You should be more than okay when translating 5000 words though. Will push a new version soon and let you know when it is ready.

@B3ta Update on the errors with lengthy content. Unfortunately it's not an issue with the module from what I can see. I was able to find out where the limitation on length was down to the sentence. When the API is used to translate it returns standardized responses that returns the translated content when it is successful, or errors that give instructions on what went wrong. Unfortunately the issue we are experiencing causes the DeepL API to return nothing (null) which means that their system experiences and issue but does not respond the way that they say it should. I'm going to be sending in a support request to DeepL detailing the issue and see what they say. Will report back with updates but as of right now it looks like Fluency can only handle medium amounts of text.

I thought about building that out from the start but wanted to focus on getting the core of the module working first in a way that matched the way ProcessWire's multi-language setup works where there is a primary language and secondary languages. Including the general translation tool in the top menubar where any text can be translated from/to any language was a short term solution for that limitation. Once Fluency is able to mature and all of the bugs fleshed out then revisiting that would be an option.

I've seen translations take varying times depending on length and language. Are you letting it work for a while? I will test more with longer content and look into how DeepL handles it.

Many thanks! Let me know if anything else pops up.

@B3ta New version pushed with fixes for the [[tag]] exclusion. All excluded strings set up in the module config should work now. https://github.com/SkyLundy/Fluency-Translation/releases/tag/v0.3.1 Still need to work on the large content translation though.

That shortcode issue you're seeing was totally my fault. I will push a fix as soon as I can complete it. It will maintain the tags and all other excluded strings properly after that. 5k words! I haven't translated anything that long. As a little test I created a text file with ~5k words in it and the file size was 28kb. The API has a request limit of 30kb and this is almost certainly hitting that ceiling. I need to build out a way for Fluency to analyze the size of the translation, split it up into multiple requests to DeepL, and the re-assemble the multiple translations back into one. Unfortunately this is a hard limitation by the DeepL API so the workaround will have to come from me. I'll put that on the to-do list.

Fluency now supports the new DeepL free accounts in the latest release (v0.3.0) which also includes some bugfixes. Now live on the repo. Thank you to @Mr. NiceGuy for notifying me about the new free accounts from DeepL. Very exciting to know that anyone can use Fluency for any project or to play around with at no cost. I hadn't noticed but since Fluency was released in November 2020 DeepL's supported languages has climbed from 12 to over 25 which is really awesome. Looking forward to seeing what languages they add next. Fluency will always offer all languages that DeepL supports as it pulls this information from the API, so no module updates are ever required to take advantage of these. Thanks everyone and bugfixes and feature requests are always appreciated over on the Github repo.

Sorry I've been away for so long. Deciding to buy a house, a puppy, and then remodeling said house while under deadline on deploying a new company API at work is an example of excellent decision making haha. I didn't know there was a free API and that is awesome news. Might have been introduced since I last worked on the module. I'll be pushing these changes tonight for both the free API option as well as the JS fix. Thank you for your help @Mr. NiceGuy. @B3ta adding engines is a longer term goal, there are still some refinements to be made before I can do that. Would definitely take a PR if anyone wants to tackle that.

Appreciate the consideration! I tried to make Fluency as modular as possible so hopefully it shouldn't be too hard to add another engine. If you take a look at how DeepL works with Fluency, it exists as it's own class and Fluency doesn't know how or what happens in DeepL, and vice versa. Following that structure, you could create a Lingvanex class that Fluency sends data to. Matching DeepL method parameters when calling it and data structures it returns could make it pretty straightforward. An idea I had would be to create a bootTranslationEngine() method that is called in ready() which would determine what engine/API key is available and instantiate the proper translator class in something like $this->translationEngine. That would replace $this->deepL in the Fluency module. I would make it a goal to abstract out any references to any brand of translator in the Fluency module's logic. As long as getClientBootData() and translate() take and return the same structure of data you wouldn't even need to touch the front end. I haven't really written any code that is collaborated on before so I hope that the commenting and overall structure makes sense. If you ever have any questions or want a shortcut to figuring something I did out then message me and we can chat.

Really no problem at all. I've released a newer version since then that addresses an issue that I didn't see come up in testing previously. Just a courtesy tag in case the bugfixes end up helping someone. I haven't heard of Lingvanex before so I didn't know to look into them. Initially when I found DeepL I was really impressed by the service but needed someone else to really verify what was going on because I only speak English. I had a coworker who speaks Spanish look at the results and he was very impressed. I kind of stopped there and didn't think to look further. Overall if someone was able to verify the quality of translation of any translation then that's all that matters. I don't have any personal allegiance to DeepL, it's just what I found and what I heard about on the PW forums which also nudged me towards them. It would be interesting in the future to update Fluency to allow the dev to select a translation engine, and the way the module is built I think it could be done well. I wish I could work on that now but I just had 3 clients contact me for work and that's in addition to my day job...

Important update to the module with a new release. Recommended that all users upgrade to the latest version The module has been updated with critical fixes and feature requests fulfilled. The module has been updated to fix an issue with lengthy content that would result in failure, mostly important for CKEditor fields that include markup in the content. This was due to the admin AJAX call to the module using a GET request where a POST request should have been used to prevent running into URL length issues. Translation was pulling from the wrong location in CKEditors so it was translating the last saved content, not the latest changes made in the field before page save. Use of Fluency is now permission based as requested. After upgrading all roles who should have access must have the fluency-translate permission added The configuration screen now has a new setting for the Preserve Formatting feature which addresses the issue reported in this thread where short strings or phrases were being returned with additional/missing punctuation. (internally this uses the preserve_formatting DeepL API parameter). It is recommended that this be enabled unless you have a specific reason not to use it. Public methods have been added to the Fluency module so that translation, usage stats, and available languages can be pulled from calling the Fluency module or by using the DeepL class directly. README has been updated with documentation for usage. A new parameter has been added to both the Fluency and DeepL translate methods that allow you to pass in any DeepL API key/values with your request. If you are currently calling either the module or DeepL class directly, please read the documentation as the parameters have changed. Return values from the Fluency module now match the return values from the DeepL module for consistency. This includes passing DeepL's HTTP response for use in programmatic conditions. The module has been moved from Gitlab to Github to adhere to ProcessWire's module directory requirements for future listing. As usual, more to follow! Latest: https://github.com/SkyLundy/Fluency-Translation/archive/main.zip Paging @adrian & @xportde & @bartelsmedia as you've been Fluency's alpha testers extraordinaire.

That's interesting, I'd never actually tested that. I'll make a note to disable those translation connections in the module. Also really odd that capability doesn't exist... seems like literally the easiest translation they'd have to do haha.

Pushed the fix that removed the bd() call. Thank you for the reminder. I put some work in over the weekend on the permission feature. The new permission is named "fluency-translate" in case you want to rename yours to match the later releases. This release also formally adds the 5th Fluency->translate() argument to feed in additional API parameters. Everything is always a time crunch between holidays here, looking forward to when I can get back to working on more changes. Permission will be the next feature. Excellent to hear that it's in production! We've been using it and it's been smooth sailing.

Proper error handling is definitely on the implementation list. I guess it just got upgraded to a higher priority item now that we've seen that. For what it's worth I've been playing around with the API for months and haven't experienced that. Not downplaying the importance of error handling, just hoping that means they're correct when they say that it happens seldomly. I'll be pushing more work when I can get some time dedicated to the module. I'm slammed at work right now but I will prioritize error handling when I'm back on the project. Thank you for reporting this here as someone might look for answers if this happens again, even with error handling.