Jump to content
bernhard

RockMigrations - Easy migrations from dev/staging to live server

Recommended Posts

Hi @zoeck

10 minutes ago, zoeck said:

And a second problem with an option field:

See setFieldOptionsString() method

11 minutes ago, zoeck said:

Are there any problems when you create repeaters and use fields that are created in the same upgrade?

There is no error, but my fields were not assigned to the repeater.

I guess I've never came up with any migration for repeater fields, so this might need some custom integration. PRs welcome. I don't use Repeaters at all these days 😉 

  • Like 1

Share this post


Link to post
Share on other sites
23 minutes ago, bernhard said:

Hi @zoeck

See setFieldOptionsString() method

Is this possible with the "migrate" function?

Just Added one Line on the "Upgrade" with the setFieldOptionsString -> working 😉 Thanks!

 

Migrations without repeaters 😞 thats uncool... 

Do you not use repeaters at all?

Share this post


Link to post
Share on other sites
11 minutes ago, zoeck said:

Is this possible with the "migrate" function?

No. migrate() is just a shortcut array syntax. You can always combine both:

$rm->migrate([
	'fields' => ['foo' => ...]
]);
$rm->setFieldOptionsString('foo', ...);

 

11 minutes ago, zoeck said:

Migrations without repeaters 😞 thats uncool... 

😶

11 minutes ago, zoeck said:

Do you not use repeaters at all?

No. Pages for the win. A lot easier to handle in many situations (Migrations, RockFinder, ...). But I have a custom Repeater implementation... Maybe the need for migrating repeaters will arise one day for me and maybe I'll implemenet something then if nobody that needed it earlier did submit a PR...

Share this post


Link to post
Share on other sites
On 5/28/2020 at 2:23 PM, bernhard said:

I guess I've never came up with any migration for repeater fields, so this might need some custom integration. PRs welcome. I don't use Repeaters at all these days 😉 

After I had a closer look at the repeaters, I could add the items to the repeater.
It is actually not that difficult, but you need a little workaround:

$repeatertemplate = $this->templates->get($this->fields->get("**REPEATERNAME**")->template_id);
$rm->addFieldToTemplate("proj_milestone_date",$repeatertemplate);

For each repeater a (hidden) template is created, you just have to find the template from the repeater and then add the field via RockMigrations 🙂 
It is also possible to add several fields with "addFieldsToTemplate", setFieldData is also possible.

  • Like 1

Share this post


Link to post
Share on other sites
On 5/19/2020 at 4:59 AM, bernhard said:

Would be great to hear your experience then. Is @LostKobrakai's module really obsolete or does it still have its place?

Just to throw an opinion here, I still use @LostKobrakai module due to the command line support and I do like the way it works and how it keeps track of migrations run, which I'm not sure it's done in RockMigrations. But I guess RockMigrations could be made to work by invoking migrations script with the php command and add some tracking, I do like a lot how the field migrations are described here. 

Share this post


Link to post
Share on other sites

Thx for that report @elabx 🙂 The other module works totally different. My migrations can be run from anywhere, anytime. That makes it a lot easier to use but of course this has drawbacks. Keeping track of with migrations have already been applied is more complicated. I'm not sure how that could be done exactly. But I have to say that since I'm using RockMigrations I have not REALLY needed such a feature. My main goal was to make the changes as easy as possible. Migrations can always be applied several times so it does not really matter if they have been applied or not.

But I'd love to have a good concept for that and since RockMigrations is totally free and open source I invite anybody to contribute and improve it 🙂 I have lots of other more important things to do.

Command line support sounds interesting though. But I'm not sure if that would bring a huge benefit compared to the tracy console. And even more I wonder if it might make a lot more sense to have a command line module that can be used by any other module. Not one that is locked to RockMigrations...

Share this post


Link to post
Share on other sites
1 hour ago, bernhard said:

But I'd love to have a good concept for that and since RockMigrations is totally free and open source I invite anybody to contribute and improve it 🙂 I have lots of other more important things to do.

Didn't want to imply your module needs anything or that it is "missing" something or that I was asking for a feature, I apologize if that's how it came out! Just wanted to give an example of an exception to the norm in case anyone finds this thread and of course, I agree it's the mission of all of us to contribute to open source and I must say this community has inspired me a lot to thrive into that direction rather than just consuming the work of others but also contributing, first step is to solve the very serious personal time management issues I have and just force me to do get  open source done lol.

1 hour ago, bernhard said:

Command line support sounds interesting though.

So basically how it's useful for me is that, I keep a bunch of sites on one server and they all share template files and fields configuration. I keep files in sync using git and git worktrees, and with a shell script I make merges from the dev branch into the rest of the branches, including the migration files (each site is a branch). Then, if I need to make an update that involves any adding/removing fields I also have another shell script that goes on every site directory and runs a specific migration. It saves me a lot of time to run this for the 50+ sites I manage, I can very easily see if an error occurred, if migration was already run, if it didn't exist (then it means that site is now correctly wired up to the git repo), etc. 

Probably this is the least likely scenario of most PW users lol, as I have the feeling that most of PW users build heavily customized sites, for very specific requirements. 

  • Like 2

Share this post


Link to post
Share on other sites
1 hour ago, elabx said:

Didn't want to imply your module needs anything or that it is "missing" something or that I was asking for a feature, I apologize if that's how it came out! Just wanted to give an example of an exception to the norm in case anyone finds this thread and of course, I agree it's the mission of all of us to contribute to open source and I must say this community has inspired me a lot to thrive into that direction rather than just consuming the work of others but also contributing, first step is to solve the very serious personal time management issues I have and just force me to do get  open source done lol.

Didn't take that as offense at all 🙂 Just tried to make clear that I have already thought of that but I don't plan to implement it any time soon. But if anybody would be up for it, I'd be more than happy to assist or to pull the changes 😉 Sorry if that was not clear.

  • Like 1

Share this post


Link to post
Share on other sites
12 hours ago, elabx said:

Probably this is the least likely scenario of most PW users lol, as I have the feeling that most of PW users build heavily customized sites, for very specific requirements.

I'm sure it's the less used path, but a CLI is useful for anybody automating deployments either using some kind of CI/CD or manually called scripts.

13 hours ago, bernhard said:

The other module works totally different. My migrations can be run from anywhere, anytime. That makes it a lot easier to use but of course this has drawbacks. Keeping track of with migrations have already been applied is more complicated. I'm not sure how that could be done exactly.

The goal of my migrations module was to result in reproducable db state, so automation is possible. If migrations can be run arbitrary times it's not going to do that – e.g. a field created and later deleted is not the same as a field deleted and later created. Therefore migrations have a fixed order (by timestamp) and a db table keeps knowledge about which migrations were already applied before.

  • Like 2

Share this post


Link to post
Share on other sites

v0.0.11 adds support for repeater matrix fields 🙂 

Thx to @aComAdi of https://www.a-commerce.ch/ for sponsoring this update! 😎

$rm->setMatrixItems('your_matrix_field', [
  'foo' => [
    'label' => 'foo label',
    'fields' => ['field1', 'field2'],
  ],
  'bar' => [
    'label' => 'bar label',
    'fields' => ['field1', 'field3'],
  ],
]);

 

  • Like 5
  • Thanks 2

Share this post


Link to post
Share on other sites

v0.0.19 adds support for RepeaterFields @zoeck

$rm->migrate([
  'fields' => [
    'my_repeater_field' => [
      'type' => 'repeater',
      'label' => 'This is my great repeater field',
      'repeaterFields' => ['title', 'body', 'images'],
    ],
  ],
]);

 

  • Like 2
  • Thanks 1

Share this post


Link to post
Share on other sites
55 minutes ago, bernhard said:

v0.0.19 adds support for RepeaterFields @zoeck

Thanks @bernhard 🙂 nice Update!

Share this post


Link to post
Share on other sites

hello

@bernhard Thanks a lot for this dreamy, powerful and simple migrations module 😀 It's great and I begun to use it with succes !

I have however one problem 🤪 :

  1. I created a module and I use the auto-update system from RockMigrations (0.0.1.php). All is OK with the install and the uninstall. I create a 0.0.2.php and update the version of the module ('version' => '0.0.2'). But when I refresh modules from Processwire, my module update in 0.0.2, but no RockMigrations is executed.
  2. When I debug it with Tracy Debuger, I have many errors like this :
    array_key_exists(): Using array_key_exists() on objects is deprecated. Use isset() or property_exists() instead on line: 171 in /var/www/html/wire/core/WireData.php
    and another error like this one
    Maximum function nesting level of '256' reached, aborting! on line: 1733 in /var/www/html/wire/core/Wire.php

Any idea to resolve it ? I tried to increase the maximum function nesting level, but it's the same... 🤯

Thanks for your help

  • Like 1

Share this post


Link to post
Share on other sites
9 hours ago, sebr said:

@bernhard Thanks a lot for this dreamy, powerful and simple migrations module 😀 It's great and I begun to use it with succes !

Hi @sebr, happy to hear that 🙂 

9 hours ago, sebr said:

I created a module and I use the auto-update system from RockMigrations (0.0.1.php). All is OK with the install and the uninstall. I create a 0.0.2.php and update the version of the module ('version' => '0.0.2'). But when I refresh modules from Processwire, my module update in 0.0.2, but no RockMigrations is executed.

9 hours ago, sebr said:

Any idea to resolve it ? I tried to increase the maximum function nesting level, but it's the same... 🤯

Unfortunately, no. And I have no time to investigate. I can only share my experience: In my setups/modules I tend to use one single migrate() method now. Instead of having a migration for every version this approach just makes sure that after running migrate() you end up with the setup you want (fields, templates, pages). This is very easy using the RM migrate() method:

$rm->migrate([
  'fields' => [
    'foo' => ['type' => 'text'],
  ],
  'templates' => [
    'foo' => ['fields' => 'foo'],
  ],
]);

Lets say in the next version I need a new field "bar", I'd just add it to that migration:

$rm->migrate([
  'fields' => [
    'foo' => ['type' => 'text', 'label' => 'Foo Label'],
    'bar' => ['type' => 'text', 'label' => 'Bar Label'],
  ],
  'templates' => [
    'foo' => ['fields' => 'foo', 'bar'],
  ],
]);

This makes it really easy to grasp the setup, because you have everything in one place and you don't need to go through every migration and try to understand the whole picture. The example above with the "old" approach would be:

$rm->setFieldData('foo', ['label' => 'Foo Label']);
$rm->createField('bar', ['label' => 'Bar Label']);
$rm->addFieldToTemplate('bar', 'foo');

Not to forget that with the new approach you can diff your changes:

image.png.b8befd65a0be8a75290cf967bf110b0c.png

Of course this comes with the drawback of a little less control and it can get tricky when multiple modules to migrations and one depends on the other. But then you can still split migrations up or do necessary changes upfront. Also removing things needs some extra lines of code, but still very easy ones:

$rm->deleteField('bar');
$rm->migrate([
  'fields' => [
    'foo' => ['type' => 'text', 'label' => 'Foo Label'],
  ],
  'templates' => [
    'foo' => ['fields' => 'foo'],
  ],
]);

image.png.5dd9dfa530d712985127f3654d3205b6.png

On uninstall I just do all the cleanup and that's usually as easy as deleting all templates and fields (because all pages get deleted automatically).

I've never ever needed to run a downgrade method. I usually have a cleanup migration that just removes fields or templates that I created during dev and maybe even pushed to the live server and then I change something and dont need those fields any more (like the "bar" field above).

RM is still not perfect. I had a chat with horst and I'd like to have logging and test-run capability, but for now this is what we have and it is saving me a ton of time and headache already and I can really not understand how anybody can work with ProcessWire without RM 😄 

  • Thanks 1

Share this post


Link to post
Share on other sites

Thanks for this response. Yes, I also think this is a good solution : just one migrate call. I my case, downgrade is not necessary. The only risk I take concerns manual changes that could be done directly under Processwire. Have a nice day !

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

×
×
  • Create New...