LostKobrakai

Migrations

51 posts in this topic

Hi @LostKobrakai, I'm trying to use Migrations module, but I'm not sure how to start. What is you workflow when creating a migration? How do you test it before it is complete. Consider I have quite complex migration which adds a field (page reference type) and a tree of pages which references to. I think I'm not able to create the migration without bugs in one go. I would like to try it during development. The workflow which comes to my mind is I make a change, backup database, run migration, check admin changes, restore database, add other change to migration etc. Yes I should update the downgrade function simultaneously and instead of restoring the database restore the migration, but there is still high possibility of bugs during restore. What do you recommend? Thanks

Share this post


Link to post
Share on other sites

There's no way to test a migration beyond just using it. That's why I feel the downgrade functionality is so important. It enables you to quickly iterate on things missing. Also keep migrations small and focused on a single concern. 

Personally I don't even worry about backing up my database just for running a single migration. I'm using CronjobDatabaseBackup, which does a backup every day or so and I've not restored once of of them by now. There's not really to much, where you would destroy something not quickly fixable via the admin backend. For more involved changes, where lot's of pages are affected I also like to split things up in multiple migrations, which I can run manually – checking things after each migration.

After a few initial migrations written the potential of bugs will also considerably drop for everything "standard". E.g. creating a field with a FieldMigrations is essentially just setting some properties. The hard stuff is already taken care for. If you make errors there it's a matter of deleting the field and rerunning the migration and everythings fine again.

2 people like this

Share this post


Link to post
Share on other sites
1 hour ago, Richard Jedlička said:

check admin changes

Hi, You might also want to compare the before and after state of the database, at least before you feel confident you are doing what you intend to do.

I've already shared a simple but useful BASH 4 script to compare databases, you might also find it useful:

 

1 person likes this

Share this post


Link to post
Share on other sites

I've just tagged 0.3.0-RC1 on Github.

I've replaced league/climate with symfony/console to allow for more advanced parameter parsing => nicer commands. Documentation on the new cli commands by now can be found running migrate or migrate [command] -h.

The commands migrate/rollback do now support multiple arguments from the cli as well as via the processwire module:

  • integer: migrate/rollback that number of files
  • filename (/path, but does fallback to filename comparison)
  • classname
  • * : migrate/rollback all ("*" in the terminal)

In addition to that does migrate also have an option to only use "latest" migrations, which are the ones newer than the latest migrated file instead of all files not migrated.

Other changes:

  • Made the mysql table name lowercase to prevent issues with case sensitivity.
  • Migration files are now managed via custom WireData/WireArray classes, which made the code quite a bit nicer than passing around just filenames.
  • Custom migration templates can be put in /site/migrations/templates/. These can be created via migrate create:custom -t [type].

I'd appreciate it if some of you guys could take a look at it if there are any issues I've missed. 

4 people like this

Share this post


Link to post
Share on other sites

I just made the 0.3.0 version the new stable version. I've also added 0.3.1-beta with a new migration type: AccessMigration. This migration is only meant to change access rules for templates like shown below. List templates with changes, prepend a + or - if the useRoles setting does need to change. For those templates then list all roles which have changes (can be none) and supply which types of access should be added or removed.

<?php

class Migration_ extends AccessMigration {
  public static $description = "Update template access rules";
  
  protected function getAccessChanges() {
    return [
      '-blogpost' => [], // Remove useRoles from blogpost template
      '+basic-page' => [ // Add useRoles to basic-page
        'subscriber' => ['+view'],
        'editor' => ['+view', '+edit'],
        'admin' => ['+view', '+edit', '+create'] // +create does require +edit
      ],
      'tag' => [
        'subscriber' => ['-edit'],
        'admin' => ['+full'] // view, edit, create and add
      ]
    ];
  }
}

Edit: Had to remove the automatic +edit for +create, otherwise it's not clear to what to revert on rollbacks.

5 people like this

Share this post


Link to post
Share on other sites

Hi @LostKobrakai ! Migrations are a great idea. But i had a trouble while trying to use it.

My setup: 

  • PW 3.0.55
  • Uikit admin theme

Tried to create a test migration and, after defining its type and description, got a PHP error:

Error: Call to undefined function wirePopulateStringTags() (line 70 of  \site\assets\cache\FileCompiler\site\modules\Migrations\Migrations.module) 

Need help. Thanks in advance!

Share this post


Link to post
Share on other sites

That's strange, this should normally be handled by the file compiler (and it works for my few pw 3 installations). 

Share this post


Link to post
Share on other sites
On 2017-3-15 at 4:46 PM, theoretic said:

Hi @LostKobrakai ! Migrations are a great idea. But i had a trouble while trying to use it.

My setup: 

  • PW 3.0.55
  • Uikit admin theme

Tried to create a test migration and, after defining its type and description, got a PHP error:


Error: Call to undefined function wirePopulateStringTags() (line 70 of  \site\assets\cache\FileCompiler\site\modules\Migrations\Migrations.module) 

Need help. Thanks in advance!

I'm also seeing this, did you find a workaround?

EDIT: Prepended the ProcessWire namespace to the wirePopulateStringTags method which seems to have fixed the problem for now.

Share this post


Link to post
Share on other sites

Yeah, the naming for those snippets do not match the style of the migrations generated by the module. They are rather meant to be copied over to existing migrations in your project.

Edit: I've appended this to the README in the snippets repo.

Share this post


Link to post
Share on other sites

Hi LostKobrakai! Great module! I'm using it to maintain a series of sites using very similar features. I didn't want to be creating, configuring fields accross ten installations.

Could it be possible to create a shell script that runs migrations across multiple ProcessWire installations? Let's assume all installations are running on the same server, with multiple docroots for each PW install.

Share this post


Link to post
Share on other sites

You can use the cli component of the module to run migrations from the shell. So you could easily add this to a custom script iterating through your projects. I'm letting my Continuous Delivery tool run migrations when deploying.

3 people like this

Share this post


Link to post
Share on other sites
On 29/06/2017 at 9:53 AM, LostKobrakai said:

[...] Continuous Delivery tool [...]

What tool are you using?

Share this post


Link to post
Share on other sites

Hi again!! I am planning to run Migrations loading multiple PW instances this way ,with a script running on the shell . I am getting the issue of the classes not getting loaded due to the way the path is specified (or at least this is the assumption) I changed it to be relative and everything seems to work, would this break something that I might not be foreseeing? 

Share this post


Link to post
Share on other sites

If it's working then there shouldn't be any issue with it, but to be honest I'd personally prefer to simply use some.sh file. Just let it move though project folders and use the module's cli (e.g. migrate run -l) to invoke migrations separately. I'd be far to worried about the multi instance stuff mixing up something somewhere and potentially affecting all of the projects at once.

1 person likes this

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!


Register a new account

Sign in

Already have an account? Sign in here.


Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.