RockMigrations - Easy migrations from dev/staging to live server

[bernhard 5,039]

Well... I've read your post 3 times and think I understand what you mean.

I don't think that there is any common definition what a migration exactly is. There are too many different scenarios where the term can be used: https://en.wikipedia.org/wiki/Migration

I'm mainly using it in my modules nowadays with the migrate() function to get consistent field/template/page setups that I can use and quickly change in the codebase that is version controlled:

$this->rm()->migrate([
  'fields' => [
    'foo' => [
      'type' => 'textarea',
      'label' => 'Text1',
      "inputfieldClass" => "InputfieldCKEditor",
      "contentType" => 1,
    ],
  ],
]);

This could simply be changed and committed:

$this->rm()->migrate([
  'fields' => [
    'foo' => [
      'type' => 'textarea',
      'label' => 'Text1',
      "inputfieldClass" => "InputfieldCKEditor",
      "contentType" => 1,
    ],
    'bar' => [
      'type' => 'textarea',
      'label' => 'Text2',
      "inputfieldClass" => "InputfieldCKEditor",
      "contentType" => 1,
    ],
  ],
]);

Then simply run a git pull on the server and I have the same setup as locally.

[benbyf 697]

I guess in that case its solving the big content VS config divided present in all CMS's. But yes, i am cyrptic it seems. I guess my point was if I dont need the auto upgrading and downgrading that the module provides then I could not use it and simply write some template and field php code that you can cool as a "migration" as your module is a wrapper for quickly doing those tasks (exlcuding the uping and downing).

In an ideal world the changes would be recorded in situ (no need to write them) and migrations would be exported to file to added to version control that can be ingested by any other PW install running the module. (I'm just all about writing less)

[bernhard 5,039]

9 minutes ago, benbyf said:

In an ideal world the changes would be recorded in situ (no need to write them)

I think everybody who has ever used the macro recorder of msoffice/vba knows that this world is not ideal 😄 But I think I know what you are talking about - I've been there 😉 

 

[benbyf 697]

HA! you do know what i mean then!!! 🙂 well I'll shut up then.

[Sascha Nos 3]

On 7/22/2020 at 11:54 PM, sebr said:

I created a module and I use the auto-update system from RockMigrations (0.0.1.php). All is OK with the install and the uninstall. I create a 0.0.2.php and update the version of the module ('version' => '0.0.2'). But when I refresh modules from Processwire, my module update in 0.0.2, but no RockMigrations is executed.

Hey there, experiencing the same issue. Is there any solution?

Thank you so much! Sascha

[Sascha Nos 3]

The migration gets only executed when the module is loaded. You need to add

'autoload' => true,

to your module.

[Richard Jedlička 87]

Hi @bernhard, thanks for a great module. I like the migrate function for declarative definition of PW setup.

1) I saw in your example you are using ModuleName.config.php files to store the config for migration. I also saw the in other PW examples the file with the same name could be used for ConfigurableModule type of module, but as I understand it has different meaning. I know I can name the file as I want but Is it a good practice to use this file for migration (and show it in examples)? 

2) Could you please add ability to set different name of directory with migrations? I'd like to use just migrations instead of RockMigrations. 

3) It would be great for the module to be easily installable by composer, it is quite easy, see https://processwire.com/blog/posts/composer-google-calendars-and-processwire/#composer-for-module-developers

 

[bernhard 5,039]

Hi @Richard Jedlička, thx for your message 🙂 

2 hours ago, Richard Jedlička said:

1) I saw in your example you are using ModuleName.config.php files to store the config for migration. I also saw the in other PW examples the file with the same name could be used for ConfigurableModule type of module, but as I understand it has different meaning. I know I can name the file as I want but Is it a good practice to use this file for migration (and show it in examples)? 

Thx, that's a good point! I've added a new example file of how I am using RockMigrations nowadays that should be really helpful to get started with the power of RockMigrations and building easy, fast and reusable modules: https://github.com/BernhardBaumrock/RockMigrations/blob/master/examples/MigrationsExample.module.php

2 hours ago, Richard Jedlička said:

2) Could you please add ability to set different name of directory with migrations? I'd like to use just migrations instead of RockMigrations. 

You might not need this feature any more once you saw the new example file, but anyhow it was an easy update and I've made the getMigrationsPath() method hookable so you can use whichever folder you like 🙂 

2 hours ago, Richard Jedlička said:

3) It would be great for the module to be easily installable by composer, it is quite easy, see https://processwire.com/blog/posts/composer-google-calendars-and-processwire/#composer-for-module-developers

I'd be happy to get a quick tutorial of how all that works and what the benefits would be and then I can take that into account. PRs are always welcome.

[Richard Jedlička 87]

Thank you for quick reply! I will take a look at it.

1 hour ago, bernhard said:

I'd be happy to get a quick tutorial of how all that works and what the benefits would be and then I can take that into account. PRs are always welcome.

I see composer installation really useful because I've just specified all dependencies in composer.json file and don't have to commit modules into my site's source code. So, when I deploy the site to production the automated script easily install all dependecies. All I have to do manually is to install the modules via admin (maybe this could be automated as well) or refresh if they are already installed. I've even created a "wrapper" composer package for processwire (see https://github.com/uiii/processwire), which will install like a regular installation, or upgrade the existing. So my site's source code only contains the site folder (without modules) and composer.json (and composer.lock) and this is enough basis to setup the whole running site by single command:

composer install

 

[bernhard 5,039]

10 hours ago, Richard Jedlička said:

All I have to do manually is to install the modules via admin (maybe this could be automated as well)

This is really easy either via RM or via default API:

$modules->get('TracyDebugger');
// or
$rm->installModule('TracyDebugger');

10 hours ago, Richard Jedlička said:

I see composer installation really useful because I've just specified all dependencies in composer.json file and don't have to commit modules into my site's source code.

I'm using a GIT setup with submodules. So all I have to do is

git submodule update --init --recursive

That way I always have my modules under control of GIT and can efficiently commit new changes from any project that I'm currently working on 🙂 

But I went ahead and added support for rockmigrations via composer: https://packagist.org/packages/baumrock/rockmigrations

Would be interested to hear if everything works!

[Richard Jedlička 87]

8 minutes ago, bernhard said:

This is really easy either via RM or via default API:

Ah, good!

8 minutes ago, bernhard said:

I'm using a GIT setup with submodules. So all I have to do is

Yes, this is also the way. But I see composer as de-facto standard for maintaining dependencies, not only PW modules. I'm used to npm in JS world and love it, so composer is the way for me.

10 minutes ago, bernhard said:

But I went ahead and added support for rockmigrations via composer: https://packagist.org/packages/baumrock/rockmigrations

Great, thank you. One more thing you could do is to add git tag for each released version, or at least add the latest one.

[bernhard 5,039]

35 minutes ago, Richard Jedlička said:

Great, thank you. One more thing you could do is to add git tag for each released version, or at least add the latest one.

Yeah, maybe I should get used to that. But for me they do not add any benefit... Maybe you could explain why they might be important for others?

[Richard Jedlička 87]

4 minutes ago, bernhard said:

Yeah, maybe I should get used to that. But for me they do not add any benefit... Maybe you could explain why they might be important for others?

It's because, packagist automaticly creates versions of a package, so you can specify which version you want to install. Without tags you can only install dev-master version (current latest commit) which composer considers as development while tags considers as stable. With tags composer installs the latest tagged version if you don't specify any, but if there is none you have to explicitly specify dev-master if your composer json is not configured to accept it automatically. See my module https://github.com/uiii/ProcessWire-FieldtypePDF as example.

[bernhard 5,039]

Thx 🙂 Here you go: https://github.com/BernhardBaumrock/RockMigrations/releases/tag/v0.0.26

I hope I'll get used to tagging and not forget about it in the future 🙂 

 

[MrSnoozles 73]

I really love the syntax of the migrations.

I wonder though if RockMigrations could be additionally used like @LostKobrakai described:  to manage the state of templates and fields in a website and create a reproducible deployment.

For that as far as I see two parts would be essential:
  - having a global place to put site-deployments (e.g. site/assets/RockMigrations) where deployments can be put with a timestamp instead of a version
  - a way to run (and track) the run migrations during deployment (maybe as extension of wire shell?)

Having a way to track schema changes between staging and live environments and run them automated in a deployment pipeline would be a huge gain for Processwire.

 

 

[bernhard 5,039]

Thx @MrSnoozles 100% agreed. I see two possibilities:

1. Anybody providing PRs
2. Anybody funding/sponsoring further development of RockMigrations by me

I'd love to work more on my modules, but I can't do that all in my spare time 😐 

[elabx 830]

4 hours ago, MrSnoozles said:

really love the syntax of the migrations.

I wonder though if RockMigrations could be additionally used like @LostKobrakai described:  to manage the state of templates and fields in a website and create a reproducible deployment.

For that as far as I see two parts would be essential:
  - having a global place to put site-deployments (e.g. site/assets/RockMigrations) where deployments can be put with a timestamp instead of a version
  - a way to run (and track) the run migrations during deployment (maybe as extension of wire shell?)

Having a way to track schema changes between staging and live environments and run them automated in a deployment pipeline would be a huge gain for Processwire.

 

I think you're talking exactly about the Migrations module from @LostKobrakai and I believe you can use RockMigrations within it without problems. It says it's deprecated but works just fine, I'm using it on latest PW master.

4 hours ago, MrSnoozles said:

Having a way to track schema changes between staging and live environments and run them automated in a deployment pipeline would be a huge gain for Processwire.

I do this exactly with that migrations module having the migrations in version control.

[bernhard 5,039]

That's a good idea 🙂 RockMigrations was built so that it can be used easily by other modules. Why not as well by another migration module 🙂 

[bernhard 5,039]

For rapid development it is really great to trigger migrations on Modules::refresh(); As I'm using this technique all the time I've added a new method "fireOnRefresh" to v0.0.27

public function init() {
  $this->rm()->fireOnRefresh($this, "migrate");
}

public function migrate() {
  $this->rm()->createField(...);
}

Another handy update is the trigger() method that I use in combination with custom pageclasses. Custom pageclasses are awesome, but they do not trigger init() and ready() methods by default. That is a pity, because I usually want several hooks to live in the pageclass and not in the module (for example auto-publishing pages or creating random pagenames etc). Now that can simply be done in an autoload module:

// in init()
$rm->trigger("\MyNamespace\MyPageClass", "init");

// in ready()
$rm->trigger("\MyNamespace\MyPageClass", "ready");

I've also updated the example module that uses this technique 🙂