FireWire

One of the things that I have to work around in a current site is lots of dynamic content. Content in blocks may change depending on values from other pages, dates/times, or updates from external sources such as APIs. Many of these are automated via hooks. My example is events and activities. Events can become active, tickets go on sale, be cancelled, end, etc. This is a block for "Featured Activities" Activities are selected by a person editing a page. They can choose as many activities as desired but only 3 are shown on the page If the date of an activity passes, or tickets are sold out, it is removed from the featured activities (automatically via hook) When one is removed another is rotated in (automatically via hook) If there are no more featured activities (all of the dates have past, or tickets are sold out) the block should no longer be output to the page. These are featured on many pages across the site with a lot of automation and updating them manually would be a very terrible thing 🤣 I have a handful of blocks that do stuff like this. The idea is a shouldRender() method for Block classes. By default it returns true but may be overridden to control whether a block will show up on the page. <?php /** * Whether this block should be rendered. Called before a block is output to the page. * @return bool */ public function shouldRender() { return true; } I have something like this cobbled together already in my code but I have to add an if statement to all of the block views that need this. I also use it for things like image galleries that shouldn't render if the image field is empty. It would be really nice native feature 😎

@bernhard Awesome as always 😎 I don't know if you have a use case for trying the filesystem cache, but I recommend it. As it's a drop-in replacement it's a default for my projects. Sometimes just knowing it's the caching method is nice (I sleep better at night?) haha If you see any issues that pop up during testing it would be great if you could share them ahead of the release. Maybe there's a fix, maybe I need to reconsider it in project.

@bernhard I made the following change to the code and it appears to work correctly. <?php public function __construct() { parent::__construct(); $this->path = $this->wire->config->paths($this); $this->wire->classLoader->addNamespace("RockMigrations", __DIR__ . "/classes"); $this->lastRunLogfile = wire()->config->paths->logs . 'rockmigrations-lastrun.txt'; // load all constant-traits in /site/modules/*/ $file = wire()->config->paths->site . "RockMigrationsConstants.php"; if (is_file($file)) require_once $file; $dir = wire()->config->paths->siteModules; foreach (glob($dir . '*/RockMigrationsConstants.php') as $file) { require_once $file; } $this->watchlist = $this->wire(new WireArray()); // $this->lastrun = (int)$this->wire->cache->get(self::cachename); <- Move this from here } public function init() { $this->lastrun = (int)$this->wire->cache->get(self::cachename); // <- To here $this->wire->classLoader->addNamespace("RockMigrations", __DIR__ . "/classes"); $this->wire->classLoader->addNamespace("ProcessWire", wire()->config->paths->assets . 'RockMigrations'); $config = $this->wire->config; $this->wire('rockmigrations', $this); if (!wire()->modules->isInstalled('MagicPages')) $this->installModule('MagicPages'); if ($config->debug) $this->setOutputLevel(self::outputLevelVerbose); // ...etc I'm not sure if there is a need to have the last run timestamp checked in __construct() that you know of and I am not familiar with but this change seems to work for migrations and resolves the issue.

New thread from this original comment in the main RM thread. So I have WireCache Filesystem installed and it's a critical part of the back end of the ProcessWire application. This is a drop-in replacement by Ryan for the core database cache system. @bernhard from your original response: So the benefits of a filesystem cache are that it's lightning fast. It's really preferable in a lot of situations to database caches which require DB queries. It would be difficult to quantify in hard numbers but the simple fact is that filesystem reads are much faster that MySQL queries. This is negligible on local environments but in production is has a significant impact. The compatibility issue is that RM is getting the lastrun value in the __construct() before the filesystem cache has a chance to boot. Is it possible to move that to init or ready?

That would be a great feature, however I will make a case for an implementation for your module. Since the module adds the lqip property and lqip() method to the image field object it would be important to selectively disable some features of the module and not others. It's important that they're always available so that there aren't any errors or unexpected behaviors. A full module disable might cause some of those. I'll send over the PR 👍

Hey @d'Hinnisdaël. I recently tracked down a behavior that isn't a fault of your module, but where an additional feature in ImagePlaceholders would be a really big help. Since this module can affect so many fields and because it hooks into very common actions (savedField, savePageField) it would be really helpful if there was a module setting to globally disable placeholder generation. The issue I reported above ended up being with another module that was triggering placeholder regeneration for a very large group of fields due to a bug. This caused major slowdowns on the server as regeneration happened for all images in dozens of fields, many with multiple images. Up to hundreds of images in total. Regardless of the cause, having the ability to temporarily disable image generation for troubleshooting or whatever need may arise would be a very great feature. If you're open to a PR on Github, I've already written this into a local copy of ImagePlaceholders where the module is configurable and a checkbox is provided on the module config page to globally disable placeholder generation. Would be happy to contribute if you're interested. Thanks again for the great module!

Thanks for sharing, good to know. I'll keep an eye out to see if this happens more.

I am sporadically seeing this issue and it's difficult to detect why/when. When I get a chance I will take another look at how Fluency can help. @bernhard I'm still convinced that this isn't a problem with RPB and the measures that you/we have worked to to address this have been very robust. RPB is at no fault if changes to Inputfields are not properly being signaled by a class change or event. So the fixes in RPB seem to work partially but there are edge cases- why though I am not 100% sure of. The following is my best guess (we shall see for sure). I (pretty recently) found out that jQuery events are completely isolated from vanilla JS events. Events dispatched from vanilla JS do not trigger listeners in jQuery and vice versa. jQuery hasn't been part of my workflow for many years and dispatching jQuery events is an extra step that modules that do not use jQuery must take when necessary. Another native behavior of vanilla JS is that programmatically setting the value of a field does not trigger a change event, which is the reason all of this matters to begin with, so RPB and PW Inputfield listeners will not be triggered. For the record, Fluency has only ever dispatched vanilla JS events. Rather than try and track down why some changes are caught by ProcessWire/RPB, I'm going to take another crack at a solution. @Tiberium to bypass this you'll need to translate a field then manually click through the language tabs and add something like a trailing space to the content in each. ProcessWire trims leading/trailing spaces on save so this won't affect your content. This is annoying but it ensures that the changes to the field content are registered by both ProcessWire and RPB via a UI interaction. Thanks to all for your patience! Improving software one bug at a time...

@bernhard Sounds good! To add to the greater conversation, here's how a .env file approach would work. A .env file is created for your local development. A .env file is created on a staging server if one is used. A .env file is created for the production server. Each file exists where they will be used and the values match the requirements for the application to run in that location. Since the .env file for a project is not committed to the repository, a commonly adopted way to document what is needed in it is to create a .env.example file that contains all of the variables used by the application without values. This is unrelated to @bernhard's reply but I didn't share some ProcessWire specific information in my original post that readers- whether choosing to use .env or not- may want to keep in mind. .env files are part of a secrets management strategy. Sensitive values should be stored in a separate location, like a password manager. The config.php file is a sensitive document. ProcessWire provides direction on securing the file on your server, however this doesn't account for workflows like Git or sharing config.php in collaborative situations. Sensitive values committed to a repository (even if it's private) must be considered compromised. It's not just about DB credentials. Here's an example: /** * Installer: User Authentication Salt * * This value was randomly generated for your system on 2023/02/22. * This should be kept as private as a password and never stored in the database. * Must be retained if you migrate your site from one server to another. * Do not change this value, or user passwords will no longer work. * */ $config->userAuthSalt = "c187bd9fx935ca861f43da592475fe6afa20a63"; Ryan's note says "private as a password", and that's real. Your auth salt must remain the same regardless of where your code is being run so it is a universally sensitive secret whether local, staging, or production. This unique value is part of how ProcessWire hashes passwords in the database. Having a config.php file committed in a repo with this value means that your site/app is now insecure. Removing it from your repo and pushing the change will not make it secure since it's in the commit history. So now you're faced with the situation of storing that value in addition to your database credentials which change depending on where your site/app is running. That's makes a case for a .env file or a secure solution like @bernhard is developing for his workflow implementation.

@bernhard Glad you found it useful! Yes. The logic behind .env files is that they are tied to the environment/server, not the application, so 1 environment = 1 .env file. Think of it as the store of values necessary for your application to run in a given environment. Your application doesn't have to care where it's being run, it just knows that the necessary values will be provided and everything works. It would overwrite the cached file since the Env utility follows the general single-file usage of .env files Out of curiosity, how would multiple .env files be useful?

@BillH Good thinking. I've used this before but didn't think to try that in this case. Thanks!

Had a client attempt to upload a large (large) image file that ended up causing issues. These are 7-9mb files that caused memory issues. I deleted the images but this error is persistent when there are no images visible in the image field when editing the page: I've since re-uploaded the images after resizing them to a reasonable file size but this error still persists. I'm sure I could dig into the database/file system to remedy this but it would be really nice to have a ProcessWire way to deal with this. It would help with this instance and automate future issues. Any ideas on how to clear a stubborn problematic image/file?

No problem at all, thank you for asking detailed questions and providing extra information. It helps troubleshoot! Did you install this module for the first time when you saw these issues or did they happen during a module upgrade? There were some finicky issues a couple of versions ago when upgrading that unfortunately can only be resolved with an uninstall/reinstall, but that should not happen when upgrading to recent versions. Out of curiosity, if you upgraded, what version did you upgrade from?

@snck The purpose of the requests on the config page are to confirm that your API key works by making an API request to get the available languages. The only time that it will make 4 API calls on the config page is when you are first entering your API credentials. After you have entered valid credentials it will only make 2 API calls when you load the module config page. This hasn't ever been an issue before so I'm very curious why you are seeing a 429 but others haven't. Here's some additional details on why you are seeing 2 or 4 API calls for languages in Fluency when using DeepL: Fluency needs to know what languages a service allows you to translate from and what languages you can translate to. Google Cloud Translate will give you this information in 1 API call. DeepL requires you to make two API calls, one for source languages and one for target languages. Fluency is built with a modular architecture so that new translation services can be added easily. If you're interested you can compare DeepLEngine.php with GoogleCloudTranslationEngine.php. Each have the same methods that return the same data type but how it gets that data is entirely internal. To see what I mean about the 2 API calls vs 1 API call compare the getLanguages() method in each of those. 2 API calls is technically just one execution of the getLanguages() function. You should only see 4 API calls when: You first enter your API credentials and hit save You change translation engines You entered incorrect credentials and have to re-enter them After that it should only be 2 when using DeepL in Fluency. Back to your issue- 4 API calls shouldn't cause a 429. The DeepL team states that API calls should be limited to 50 requests per second. That comment is in an issue in their Node library, but the limits should still apply for all API usage. I just removed my credentials and re-added them without any issues. Are you seeing 4 API calls every time you load or save the module config page? I have to ask the obligatory security question- is there any chance that your credentials are being used by someone else or were compromised? I know that you said that you just created your API key today but you would have to be absolutely hammering the DeepL API to get a 429. I've never seen it during regular usage and it hasn't come up here in the forums by anyone who has been using the module for years. This is a tough issue to crack especially since it's happening for you but not me at the same time. Let me know if there's any additional info you can share.

Yeah, that's DeepL returning 429 as your error and nothing as the response body. I'm at a loss on how to diagnose that. Have you been able to translate content successfully in the past? Have you used up your allowable number of translated characters? This is going to have to be something on the DeepL side and I don't know if there's anything I can do to help out. Like I mentioned, I just translated using my API key and things are working alright.

@snck That may be a gamble... I think I've seen some situations where DeepL responses aren't very descriptive beyond the HTTP code that they return which can be frustrating. Unfortunately when I test my API key (free) I'm getting translations successfully. In your case the only way to get "Translation rate limit exceeded" is if DeepL returns an HTTP 429 code so that is another message that Fluency is relaying to you from DeepL and not originating with anything in the module. Whether the HTTP 429 applies in your case, I don't know. You may want to check the API key type that you have and ensure that it is valid for making translation requests. If you would like to see what DeepL is responding with directly and you aren't afraid to dig into some code then you can dump the response at L252 in DeepLEngine.php. Because the translations are made via AJAX in ProcessWire, you'll have to use the developer tools in your browser to view the response after clicking the translate button for a field. If you're using Tracy then adding bd() is the best way to see the result in your browser. Sorry I can't be of more help, there have been some instances in this thread where people have had some hiccups with DeepL. It doesn't happen very often and when you find the cause/solution for your issue here it should be smooth sailing.

@snck The issue may be related to the solution I suggested. He didn't respond after my suggestion so it is a good place to start troubleshooting. Please check this first: Please report back if this doesn't fix your issue. @bernhard I've got a hard Friday deadline for a project but as soon as I can get time after that I will work on that improvement. Sorry for the wait, didn't want you to think I forgot about you 👍

@bernhard Not sure if anyone else has seen this issue but on a fresh install RockGrid isn't detected. Tracy shows this warning: Fix: <?php // ...rest of module // use product variations? if (!wire()->modules->isInstalled('RockGrid')) { $fs->add([ 'type' => 'markup', 'value' => 'Product variations require RockGrid>=1.3.0. Please install RockGrid first.', ]); } else { // check if RockGrid is at least 1.3.0 if (version_compare(wire()->modules->get('RockGrid')->config->version, '1.3.0') < 0) { // Add ->config before accessing ->version property here $fs->add([ 'type' => 'markup', 'value' => 'Product variations require RockGrid>=1.3.0. Please update RockGrid to the latest version.', ]); } else { $fs->add([ 'type' => 'checkbox', 'name' => 'useProductVariations', 'label' => 'Enable Product Variations', 'checked' => $this->useProductVariations ? 'checked' : '', 'columnWidth' => 100, 'notes' => 'When checked, RockGrid and all related fields will be installed. Once installed, unchecking the box will not remove the module or fields, it will just not use them.', ]); } } // ...rest of module Works like a charm!

@bernhard Having the coldest glass of water in hell doesn't change the fact that you're in hell 🤣

@millipedia The Mullenweg affair has been nothing short of a three ring circus. My partner manages a website where they work and it is truly a headache, and speaking to the ecosystem, that site is on WPEngine and the experience of working with them is as bad as WP itself. Side note on WPEngine, they charge premium prices for a very not premium product. They host on shared servers, you don't even get a VPS. Their sales team is overly aggressive. My partner looped me in on an email exchange for help where they were trying to bully them into paying for the next tier of hosting which was a lot more expensive and more than what they need. WPEngine said: "With the traffic your website gets we need to make sure that your website doesn't affect the other sites on the server." So confirmed with them that they are hosting this site (for the premium money they charge) on a shared server. They said "yes". So I replied with: "Should we be concerned that traffic to other sites on your server is affecting our site? If that's the case then this is a WPEngine service quality problem and that may affect the future of hosting this site with you." They said no and backed down. It took a developer familiar with web and hosting to stop their sales team from bullying a customer. This may not be a $500 plugin that is mentioned in that article, but it absolutely fits within the same story of outlandish pricing and substandard quality in the WordPress ecosystem. The amount of abuse that developers using WordPress and the clients who have to live with it endure will never cease to amaze me.

My favorite thing about working with the Salesforce API is the feeling you get after you're done programming against it and don't have to anymore. I'm so sorry 😔. Safe travels!

@Ana To add that type of user-facing functionality would best be achieved using AJAX. I think that this is a perfect use case for htmx which works very well with ProcessWire and is probably the fastest and easiest method to get the type of behavior you're looking for. Here's a working example of what this would look like. It takes about 5 minutes and you don't even have to leave your template file. <!DOCTYPE html> <html lang="en"> <head> <title><?=$page->title?></title> </head> <body> <!-- ...the rest of your template markup --> <!-- Your button to load more pages --> <div> <button hx-get="<?=$page->url?>?content=remaining-pages" hx-target="#remaining-pages-container" hx-select="#remaining-pages"> View More </button> <div id="remaining-pages-container"> <!-- Your additional page markup will be loaded here --> </div> </div> <!-- This page list will only render if the URL has ?content=remaining-pages appended --> <?php if ($input->get->content === 'remaining-pages'): ?> <ul id="remaining-pages"> <li>Remaining Pages</li> <?php foreach ($pages->find('your=selector,goes=here') as $remainingPage): ?> <li> <h2><?=$remainingPage->title?></h2> <a href="<?=$remainingPage->url?>">View Page</a> </li> <?php endforeach ?> </ul> <?php endif ?> <script src="https://unpkg.com/htmx.org@2.0.4" integrity="sha384-HGfztofotfshcF7+8n44JQL2oJmowVChPTg48S+jvZoztPfvwD79OC/LTtG6dMp+" crossorigin="anonymous"></script> </body> </html> Explanation. Having added the htmx script tag to your document, the htmx library is available. Here's what happens: Clicking the "View More" button makes an AJAX request to the server The URL for the hx-get attribute has been appended with "?content=remaining-pages", this can be anything you prefer The conditional block for additional pages will only render if the ProcessWire $input->get->content matches the GET variable we appended to the hx-get attribute URL When the page is rendered and response sent back for the AJAX request, htmx finds the element with the id 'remaining-pages' If an element with the id of 'remaining-pages' is found, then the contents of that element are inserted into an element with the id of 'remaining-pages-container' which is specified in the hx-target attribute of the <button> element There are some other ways that you could organize your code and return the markup, that's entirely up to you and how you would like to organize that in your ProcessWire site. This works, but we can make this more efficient. In the example above, the entire page is rendered just to get the remaining pages. This may not be an issue for all websites, but if a page has a lot of fields or other features that make database calls, then there is a lot of work being done on the server to render content that won't be shown because htmx is only selecting the content inside the element with the id of 'remaining-pages'. Here's an improvement: <!-- This page list will only render if the URL has ?content=remaining-pages appended --> <?php if ($input->get->content === 'remaining-pages'): ?> <ul id="remaining-pages"> <li>Remaining Pages</li> <?php foreach ($pages->find('your=selector,goes=here') as $remainingPage): ?> <li> <h2><?=$remainingPage->title?></h2> <a href="<?=$remainingPage->url?>">View Page</a> </li> <?php endforeach ?> </ul> <?php return; ?> <?php endif ?> <!DOCTYPE html> <html lang="en"> <head> <title><?=$page->title?></title> </head> <body> <!-- ...the rest of your template markup --> <!-- Your button to load more pages --> <div> <button hx-get="<?=$page->url?>?content=remaining-pages" hx-target="#remaining-pages-container" hx-select="#remaining-pages"> View More </button> <div id="remaining-pages-container"> <!-- Your additional page markup will be loaded here --> </div> </div> <script src="https://unpkg.com/htmx.org@2.0.4" integrity="sha384-HGfztofotfshcF7+8n44JQL2oJmowVChPTg48S+jvZoztPfvwD79OC/LTtG6dMp+" crossorigin="anonymous"></script> </body> </html> This is the exact same code as above, but here's what we did differently this time. The markup for the additional pages that only renders if 'content=additional-pages' is present has been moved above all of the other markup in the template After the closing <ul> tag and within the <?php endif ?> statement, we've added a return statement that prevents ProcessWire from rendering the rest of the page Depending on how "heavy" your template is, this could make the results appear to the user faster and very little work was needed on our part to do that. There are things that you can explore to make use of other ProcessWire features and improve the readability and/or reusability of your code Rather than using a GET variable, you could choose to use URL segments in ProcessWire to create and respond to a more natural URL You could use the render feature of FileTools in ProcessWire to store your remaining pages markup in a separate file You could use a GET variable such as "count=6" to set the number of pages to return rather than hard coding it Those are entirely optional, the example above is all you need to implement the feature you are looking for. Some notes on htmx If you use a build tool and npm, you can add htmx to your project as a package rather than the <script> tag htmx offers an easy way to create a 'loading' animation to let your users know that the button is retrieving the remaining pages while the AJAX request is being processed Hope this is useful for your project! The code example really did take only 5 minutes (maybe less!), without one line of JavaScript. If you prefer using a JavaScript oriented approach with JSON and markup such as the <template> tag as @TomPich mentioned, the same thing could be achieved using Alpine.js which is an easy way to add enhancements to your templates without the need for a full front end framework (just another personal favorite).

@JayGee Happy to hear the module is useful! I haven't used page breaks before so I hadn't tested that. Since diving further into htmx driven FormBuilder forms outside of this module I've recognized a few places where it can be improved. Another item that needs to be handled is file uploads. Work keeps me from doing things so I won't be able to handle this immediately. How critical is the issue for your use case? MarkUp Regions shouldn't be a problem so no worries there AFAIK.

@Mike-it some steps: Double check that you've picked the correct "Free" or "Pro" subscription type on the module config page. This is an issue with a DeepL account, API key, or a Free/Pro setting. If you receive that error again, next to the "check credentials" there is a +2, click on the chevron-down and see what the additional message is that may be hidden. There is only one place in the module that can return that error message and it is based on the HTTP response status. So in that case, DeepL is indeed returning an HTTP 403 and Fluency is just relaying the information. The error message you're seeing is the "polite" message from Fluency. Any errors returned by a translation service are logged so there may be a more specific message in the fluency-engine log if DeepL provided one.

@Ivan Gretsky I'm stuck under a lot of work and deadlines. If you are able to test and provide any info that would help me debug that would make a fix much easier on my part. See if there are any logs generated, dev tools console errors, or share any exceptions you're receiving when attempting to translate. Is your issue related to the potential namespace issue that is noted under the comments of the PR? I missed the Github notifications on that so I'm just becoming aware of it now.