snck

Hi @bernhard — first of all: thank you again for RockMigrations. It has become one of those “can’t-live-without” tools in my ProcessWire projects and makes CI/CD + deployments incredibly smooth. Why I needed a workaround (shared hosting reality) In this client project I migrate pretty much everything via RockMigrations: fields, templates, roles/permissions, installed modules — and also some pages/permissions that are created dynamically based on constants/config values that change from time to time. That works great conceptually, but on a very slow shared host it led to painfully long migration runs even when there was nothing to change. A lot of my migrations are “dynamic config” style and get evaluated every deploy, which was costing me minutes per run (and that adds up quickly with multiple deploys/commits per day). A tiny helper module: hash-based skipping for config migrations I wrote a small helper module (not meant to be more than a little utility) that helps specifically with this scenario: Build a migration $config array (can include dynamic parts). Normalize + hash the config. If the stored hash for a key matches, skip running the migration. If it differs, run it and store the new hash. So migrations only execute when the effective config changed. How I use it (complete example) This is how it’s used in my migrations (example shortened, but complete in terms of usage): <?php namespace ProcessWire; use RockMigrations\MagicPage; class AuthorPage extends Page { use MagicPage; public function migrate(): void { /** @var RockMigrations $rm */ $rm = $this->wire->modules->get('RockMigrations'); $speed = $this->wire->modules->get('DlfMigrationSpeedUp'); $config = [ 'fields' => [ 'firstname' => [ 'label' => 'First name', 'type' => 'FieldtypeText', ], 'lastname' => [ 'label' => 'Last name', 'type' => 'FieldtypeText', ], ], 'templates' => [ 'author' => [ 'fields' => [ 'title' => [ 'columnWidth' => 50, 'label' => 'Display name', ], 'firstname' => [ 'columnWidth' => 50, ], 'lastname' => [ 'columnWidth' => 50, ], ], ] ] ]; $key = 'template:AuthorPage'; if ($speed && !$speed->shouldRun($key, $config)) { // unchanged -> skip } else { $rm->migrate($config); if ($speed) $speed->remember($key); } } } Important limitations (when this does not apply) This only works if the migration outcome is fully determined by the config you hash. It’s not a good fit if relevant parts can be changed outside code, for example: field/template settings changed in the PW admin GUI, manual permission tweaks, any DB drift that is not represented in the hashed config. In my case it’s safe because I enforce “everything via RockMigrations/API” for this instance — but it’s definitely not something I’d blindly recommend for every setup. Performance impact (real numbers) On that shared host, some of my “dynamic config” migrations were extremely slow even when they effectively had nothing to change. Example excerpts from one run before the helper (I’m anonymizing the real class names here): Before: TemplateXPage::migrate() 53.434s TemplateYPage::migrate() 29.347s TemplateZPage::migrate() 20.431s …many more similar steps… Total runtime in that log was ~226s. With the helper enabled, unchanged migrations are skipped and return immediately: After (unchanged config): TemplateXPage::migrate() 0.000s TemplateYPage::migrate() 0.000s TemplateZPage::migrate() 0.000s …everything essentially no-op… Total runtime (from the log): ~0.72s. That’s a night-and-day difference for repeated deployments where the effective config didn’t change. Code (module) on Gist I didn’t want to paste the whole module here; the code is in this Gist: https://gist.github.com/fsnck/cf7799c1f761681cc5f0dea13e3b802a If anyone has thoughts on edge cases or improvements, I’m happy to discuss. And again: thanks @bernhard — RockMigrations is awesome.

@erikvanberkum I (successfully) upgraded a website from RM 5.X to 6.9.0 and 7.1.0 in the end and had zero problems. But no single ProcessWire instance I am working on with RockMigrations has a file site/modules/MagicPages/MagicPages.module.php. In all my installations it is site/modules/RockMigrations/MagicPages.module.php. As MagicPages are part of RockMigrations, this should be the reason for the error you are experiencing. I would delete the site/modules/MagicPages directory as it seems redundant to me and do a modules refresh. Cheers, Flo

I installed the module for the first time (v2.1.1), no upgrade process involved. 😬

@FireWire Thank you so much for your explanation and patience. I got four requests every time I loaded the settings page. I uninstalled the module, reinstalled it, entered the same settings, saved them (2 times) and out of a sudden it worked. I am able to configure the languages and translate. Great experience so far! 👍 One quick observation: I am using @bernhard's RockPageBuilder. In a gallery block that contains multiple images (Pageimages field) I wanted to translate the descriptions. Translating them works as expected (translation is shown in the field), but after saving (clicking Save button or pushing Return key) the field is empty again (translation gone). If I change the field's value (before or after translating), the translation is saved as expected. I noticed that whenever I manually change something in the description field, .InputfieldStateChanged class is added to the inputfield wrapper (#wrap_Inputfield_rpb_gallery_images_repeater1613 in this case), but not if I just click the Translate link. I suspect that the change in the description field is somehow not detected correctly and not saved in consequence? Maybe there is a way for Fluency to trigger the detection of that change?

@FireWire No, never used Fluency or the DeepL API before. The account has been created today, the key is fresh and I could successfully use it with curl from the shell without problems. I do not really understand the goal of the requests that are made on the settings page. As I get status 200 for 3 times, the connection to the API looks fine to me. This looks like requests 2 and 4 are identical (same endpoint, data and method). As they are made immediately after another a rate limit error is to be expected. What is the outcome or behaviour you would expect?

@FireWire Thank you! I tried that. I get the following dumps: The arrays list the available languages: The request that returns null has status code 429:

@FireWire Thanks! I had already checked that. I picked "Free" (which is correct) and inserted a valid API key that has never been used before. I only get the "Translation rate limit exceeded, please wait then try again" error (see below). The fluency-engine logs show the following message: Engine: DeepLEngine /Error: RATE_LIMIT_EXCEEDED /Message: No Message /Response: null Is there any other place where I could look for more verbose information? I tested the API key and made sure it is correct: curl -X POST 'https://api-free.deepl.com/v2/translate' -H 'Authorization: DeepL-Auth-Key [MY API KEY]' -d 'text=Hello%2C%20world!' -d 'target_lang=DE' {"translations":[{"detected_source_language":"EN","text":"Hallo, Welt!"}]}%

@Mike-it Have you found a solution? I am experiencing the same problem on 3.0.246.

Hey @bernhard, I did some further research and at least came up with a workaround. As title is a global field, I added the following config migration: <?php // site/RockMigrations/templates/field-images.php return [ 'fields' => [ 'image_credit' => [], ], 'noGlobal' => 1, // prevent global fields like title from being added to the template during migrations ]; After manually removing the title field from field-images in the GUI this prevents RM from automatically adding it.

Today I noticed something strange on a client website (in dev). I have three image fields (e.g. images) that use custom fields to add a field (called image_credit). Somehow RockMigrations ins adding the title (PageTitle) field on every modules refresh to these templates (e.g. field-images) although I have no migrations in place for these fields. I cannot find any hint in the logs as well. Whenever I manually delete the field in the GUI and do a modules refresh, the field is added again. Disabling RockMigrations ($config->noMigrate = true) stops this behavior, but does of course not solve my problem. @bernhard, were there any recent changes that could be causing this? I am on RM 6.8.1. Any help is appreciated! 🙂

@Sebi, thanks for your quick reply! The server is running on Apache and using nginx only as a reverse proxy. No hidden place of logs that I can think of (checked Apache and nginx access and error logs). And except of the status code everything is indeed working as expected. Sorry, "x-original-status: 200" is my fault. I added this header temporarily in your AppApi's module while trying to track down the error.

I did some more testing and research. If I change the status code to 202 in my template, the status is sent correctly (to the browser and to the web app). My template looks like this: if (wire('modules')->isInstalled('AppApi')) { $module = $this->wire('modules')->get('AppApi'); // Check if page was called via AppApi if($module->isApiCall()){ // Output data $output = [ 'id' => wire('page')->id, 'name' => wire('page')->name, 'more' => 'stuff ...', ]; // sendResponse will automatically convert $output to a JSON-string: AppApi::sendResponse(200, $output); // 200 leading to 500, 202 working correctly! } As other AppApi endpoints that do not use AppApiPage are working, this is maybe an issue of ProcessWire changing the status code? As AppApiPage is using $page->render(), this could be a part of the problem? As soon as I open any frontend page of this PW instance with the browser I am testing with, the following requests to /api/page/... correctly send status 200. Is it needed to initialize a session to make the correct status codes work?

Hey @Sebi, I had zero problems for several months, but today a client told me that a site, that was working perfectly, suddenly stopped working. I have a SvelteKit WebApp that uses PW as API with AppApi and all other routes are working fine (status code 200, correct json) except of those called via /api/page/... The (API) webserver gives a status code 500, although outputting correct json: curl -v -H "Origin: https://domain.com" https://api.comain.com/api/page/touren * Trying XX.XX.XX.XX:443... * Connected to api.domain.com (XX.XX.XX.XX) port 443 (#0) * ALPN: offers h2,http/1.1 * (304) (OUT), TLS handshake, Client hello (1): * CAfile: /etc/ssl/cert.pem * CApath: none * (304) (IN), TLS handshake, Server hello (2): * (304) (IN), TLS handshake, Unknown (8): * (304) (IN), TLS handshake, Certificate (11): * (304) (IN), TLS handshake, CERT verify (15): * (304) (IN), TLS handshake, Finished (20): * (304) (OUT), TLS handshake, Finished (20): * SSL connection using TLSv1.3 / AEAD-AES256-GCM-SHA384 * ALPN: server accepted h2 * Server certificate: * subject: CN=api.domain.com * start date: Dec 9 14:27:01 2024 GMT * expire date: Mar 9 14:27:00 2025 GMT * subjectAltName: host "api.domain.com" matched cert's "api.domain.com" * issuer: C=US; O=Let's Encrypt; CN=R10 * SSL certificate verify ok. * using HTTP/2 * h2 [:method: GET] * h2 [:scheme: https] * h2 [:authority: api.domain.com] * h2 [:path: /api/page/touren] * h2 [user-agent: curl/8.1.2] * h2 [accept: */*] * h2 [origin: https://domain.com] * Using Stream ID: 1 (easy handle 0x14c00c600) > GET /api/page/touren HTTP/2 > Host: api.domain.com > User-Agent: curl/8.1.2 > Accept: */* > Origin: https://domain.com > < HTTP/2 500 < server: nginx < date: Thu, 09 Jan 2025 23:51:40 GMT < content-type: application/json < expires: Thu, 19 Nov 1981 08:52:00 GMT < cache-control: no-store, no-cache, must-revalidate < pragma: no-cache < x-powered-by: ProcessWire CMS < access-control-allow-origin: https://domain.com < access-control-allow-headers: Content-Type, AUTHORIZATION, X-API-KEY < access-control-allow-credentials: true < x-original-status: 200 < set-cookie: wires=krcrpn4pn6v9pc16mbqXXXXXX; path=/; secure; HttpOnly; SameSite=Lax < x-frame-options: SAMEORIGIN < x-xss-protection: 1; mode=block < {"last_modified":1729504403,"tours":[{...}]... (json is fine) I tried a lot, but there is nothing in the logs indicating a solution. Maybe you have an explanation or can give me a hint? This was working perfectly fine for more than a year and suddenly stopped working, although nothing (PHP version, PW version, code) changed in the last few months (at least nothing that I am aware of). Any help is appreciated. Thanks, Flo

Hey @bernhard, I usually use a setup where a production website and a staging website are on the same file system. I love the filesOnDemand feature because it saves a lot of disk space for the local development system and also for the staging system, but it is also very slow on sites with a lot of assets. I think it would be great to be able to specify a local path instead of a URL to a website (or in addition). Ideally, if a local path is specified, then if a file or variation (for images) is not present, filesOnDemand would first search via the specified alternate path to see if the assets are present and deliver them (without downloading or copying) directly. The assets folder of the staging system would therefore only grow when a user creates new pages or makes changes to existing ones, but not when content that exists in the production system (on the same file system) is to be accessed. I assume that filesOnDemand has so far been developed primarily for downloading assets that are only available remotely. But perhaps it could also cover this application purpose - or a new feature with a different name? Cheers, Flo

Unfortunately this strange behavior reappeared. 😞 But at least I got some more errors in rockmigrations.txt: As the (growing) indentation indeed looked like a loop to me I had another look at the releases of RockMigrations. In this case commit 476963f (feat: add refresh() before installing new module) seems to be the culprit. Commenting the refresh() in line 2454 in RockMigrations.module.php does the trick for me: // if module is not installed do a refresh // this is necessary sometimes (don't know why) $unindent = false; if (!wire()->modules->isInstalled($name)) { $this->log("Install module $name"); $unindent = true; $this->indent(2); // $this->refresh(); } else $this->log("Already installed $name"); The logs indicate that there might be something going wrong with the installation of ProcessRockMigrations? After uncommenting the line above again, the error did not reoccur. I tried to reproduce it by re-running the Github Actions, but everything seems to be working so far. @bernhard, maybe you could have another look at the refreshs and/or the routine installing ProcessRockMigrations? This might be an edge case, but one that could be prevented?