ProcessDbMigrate

[MarkE]

Season's Greetings ProcessWirers! I hope you enjoy the gift of this module, but use with care...

TLDR: This module captures changes made in the development environment so that they can be easily migrated to the live environment without needing to specify the changes or write any code. The demo below gives a brief overview.

Want to read? Read on.

One of the (few) problems with ProcessWire, in my opinion, is the lack of any native way of handling migrations. Given that PW is such a powerful tool capable of sophisticated and complex web-based applications, this is less than ideal. There is a solution, however, in RockMigrations which accomplishes a lot in a controllable way, provided you are happy to specify your database set-up in code rather than via the UI (albeit that the latest versions allow you to grab much of the required code from the UI). If that suits your need, great. Around the same time as the first versions of RockMigrations, I started developing my own UI-based migrations module, which I have been using with reasonable success for some time. I halted development of the module for a while as RockMigrations developed and I considered switching to that route.

However, I decided that my module suited me better and that a real improvement could be made if it was effectively automated so that I no longer needed to specify a migration. So that is exactly what it does: after configuring the module, you add a new migration page with ‘log changes’ enabled (which includes determining what types of objects are relevant for the migration) and work on your development system. Once you have made the desired changes (and tested them!) in the development environment, you go back to the migration page where it has magically captured the objects which have changed and listed them in dependency order. You then ‘export’ the changes, which creates json files to be uploaded to the live environment (via Git or FTP etc.), where they are then ‘installed’ to re-create the changes in the live system.

The demo below illustrates this briefly. This first demo shows the creation of a migration. The installation demo will be in the next post, because of size constraints. See post 4 for HD video.

 

There is a very extensive manual which covers all the features of the module, not just this ‘automatic’ method.

Available on github at https://github.com/MetaTunes/ProcessDbMigrate and in the modules library here.

PLEASE NOTE that this is still in 'alpha'. Do not use in production without fully testing and backing up at every stage. It is quite complex so, although I have tried hard to eliminate bugs, there will inevitably be some left!

Edited December 23, 2023 by MarkE
Now in modules library

[MarkE]

Demo part 2, showing the installation of the migration was going to be here, until I realised that the file constraint is for the thread as a whole!

You can see it here: ProcessDbMigrate video - target - 1703029659248.mp4

See post 4 for HD video.

[szabesz]

Nice work and thank for sharing @MarkE! Could you please somehow share higher-resolution videos than those above? (Maybe using YouTube?) The first one is 37MB, and the second one is only 27MB, making the text hard to read. Sure, I understand what is going on, but still...

[MarkE]

1 hour ago, szabesz said:

Could you please somehow share higher-resolution videos than those above?

Sure. Part 1 is here: https://1drv.ms/v/s!AmS0-Sk4lDz9h5tDqJwj50qq6-KEhg

and part 2 here: https://1drv.ms/v/s!AmS0-Sk4lDz9h5tS2DJsT94F92fJmw

If you want to play with it over the holidays 🤔, the best thing to do is to create a clean PW installation - ddev is great for doing this quickly - and I would recommend add ProcessDatabaseBackups and TracyDebugger. Start by following similar steps to the video.

Note that, in the video, I have 'simulated' the migration installation by taking a backup before adding new fields etc. Take another backup after creating the migration, then restore the original one, changing the database name in the module config.

And don't forget to RTM: https://metatunes.github.io/DbMigrate/help.html

Edited December 20, 2023 by MarkE
further info

[szabesz]

Thanks @MarkE!

35 minutes ago, MarkE said:

If you want to play with it over the holidays

I will! :)

[gebeer]

Thank you for this module. Great addition. Now people who prefer to work more in the GUI can manage migrations with your module. The automatic recording feature looks interesting.

[MarkE]

Now in the modules library at https://processwire.com/modules/process-db-migrate/. 

[valan]

After 10 years of PW development it looks like a break-through!😁 @MarkE thank you! I'll consider using PW again in my next prj.