RockMigrations by bernhard

The Ultimate Automation and Deployment-Tool for ProcessWire

rockmigrations.svg


See the video here:

thumb.png


Module Description

RockMigrations has an easy API to do all the things you can do in the PW backend via code. This means you can fully version control your site or app simply by adding all the necessary fields and templates not via clicking but via writing simple scripts that do that tasks for you.

The module also contains several helpers that make it extremely easy to implement fully automated CI/CD pipelines.

Wiki

Check out the WIKI for a Quickstart and Docs!

Limitations


RockMigrations might not support all external fields, especially not ProFields like RepeaterMatrix. Adding support has no priority for me because I'm not using it. If you need support for it please provide a PR or if you are interested in sponsoring that feature please contact me via PM in the forum.

But not to forget: You can still use the regular PW API to create fields and manipulate all kinds of things. It might just not be as convenient as the RockMigrations API.

Where do I find out all those field and template properties?


  1. You can edit your field or template and copy the code from there (I recommend to only copy the settings you need to make your migration files more readable): img

  2. Hover the caret on the very right of the field of the setting you want to set: img

Magic


RockMigrations does not only help you with your migrations and deployments but it also adds a lot of helpers that make developing with ProcessWire even more fun.

See WIKI for MagicPages!

Snippets

Another option that helps you get started with migration syntax is using the shipped VSCode snippets. I highly recommend enabling the syncSnippets option in your config:

// site/config.php
$config->rockmigrations = [
  "syncSnippets" => true,
];

Watching files, paths or modules


RockMigrations can watch files, paths and modules for changes. It will detect changes on any of the files on the watchlist and trigger migrations to run if anything changed.

As from version 1.0.0 (29.8.2022) RockMigrations will not run all migrations if one file changes but will only migrate this single changed file. This makes the migrations run a lot faster!

When run from the CLI it will still run every single migration file to make sure that everything works as expected and no change is missed.

Sometimes it is necessary that even unchanged files are migrated. RockMatrix is an example for that, where the module file triggers the migrations for all Matrix-Blocks. In that case you can add the file to the watchlist using the force option:

// inside RockMatrix::init
$rm->watch($this, true, ['force'=>true]);

Watching modules

You can easily watch any ProcessWire module for changes and trigger the migrate() method whenever the file is changed:

// module needs to be autoload!
public function init() {
  $rm = $this->wire->modules->get('RockMigrations');
  if($rm) $rm->watch($this);
}
public function migrate() {
  bd('Migrating MyModule...');
}

Watching files

You can watch single files or entire paths:

$rm->watch(__FILE__, false);
$rm->watch(__DIR__."/foo");

Note that you need to define FALSE as second parameter if the file should not be migrated but only watched for changes. If you set it to TRUE the file will be included and executed as if it was a migration script (see examples below).

Running migrations


RockMigrations will run migrations automatically when a watched file was changed. In case you want to trigger the migrations manually (eg after deployment) you can use the migrate.php file:

php site/modules/RockMigrations/migrate.php

Sometimes you want to work on a file and you want it to be watched for changes, but you don't want to trigger the migrations all the time. For example when working on markup or LESS. In that case you can disable automatic running of migrations either by enabling CLI mode or by calling noMigrate():

// in site/ready.php
/** @var RockMigrations $rm */
$rm = $this->wire->modules->get('RockMigrations');
$rm->noMigrate();

// in your cli script
define('RockMigrationsCLI', true);

Files On Demand


You can instruct RockMigrations to download files on demand from a remote server. This makes it possible to create content on the remote system (eg on the live server), pull data from the database to your local machine and as soon as you open a page RockMigrations will fetch the missing files from your remote server.

// without authentication
$config->filesOnDemand = 'https://example.com';

// with http basic authentication
$config->filesOnDemand = 'https://user:password@example.com';
YAML
$rm->watch("/your/file.yaml");
fields:
  foo:
    type: text
    label: My foo field
PHP
$rm->watch("/your/file.php");
<?php namespace ProcessWire;
$rm->createField('foo', 'text');

Auto-Watch

RockMigrations automatically watches /site/migrate.php and files like YourModule.migrate.php.

Working with YAML files


RockMigrations ships with the Spyc library to read/write YAML files:

// get YAML instance
$rm->yaml();

// get array from YAML file
$rm->yaml('/path/to/file.yaml');

// save data to file
$rm->yaml('/path/to/file.yaml', ['foo'=>'bar']);

Working with fieldsets


Working with fieldsets is a pain because they need to have an opening and a closing field. That makes it complicated to work with it from a migrations perspective, but RockMigrations has you covered with a nice little helper method that can wrap other fields at runtime:

// syntax
$rm->wrapFields($form, $fields, $fieldset);

// usage
$wire->addHookAfter("ProcessPageEdit::buildForm", function($event) {
  $form = $event->return;

  /** @var RockMigrations $rm */
  $rm = $this->wire->modules->get('RockMigrations');
  $rm->wrapFields($form, [
    'title' => [
      // runtime settings for title field
      'columnWidth' => 50,
    ],
    // runtime field example
    [
      'type' => 'markup',
      'label' => 'foo',
      'value' => 'bar',
      'columnWidth' => 50,
    ],
    'other_field_of_this_template',
  ], [
    'label' => 'I am a new fieldset wrapper',
  ]);
})

Deployments

You can use RockMigrations to easily create fully automated CI/CD pipelines for Github. It only takes these simple steps:

  • Setup SSH keys and add secrets to your repository
  • Create workflow yaml file
  • Push to your repo

Setup SSH keys and add secrets to your repo


To use this workflow you need to set the referenced secrets in your git repo.

Create a keypair for your deploy workflow. Note that we are using a custom name id_rockmigrations instead of the default id_rsa to ensure that we do not overwrite an existing key. If you are using RockMigrations on multiple projects you can simply overwrite the key as you will only need it once during setup:

ssh-keygen -t rsa -b 4096 -f ~/.ssh/id_rockmigrations -C "rockmigrations-[project]"

Copy content of the private key to your git secret SSH_KEY:

cat ~/.ssh/id_rockmigrations

Copy content of keyscan to your git secret KNOWN_HOSTS

ssh-keyscan your.server.com

Add the public key to your remote user:

ssh-copy-id -i ~/.ssh/id_rockmigrations user@your.server.com

Or copy the content of the public key into the authorized_keys file

cat ~/.ssh/id_rockmigrations.pub

Try to ssh into your server without using a password:

ssh -i ~/.ssh/id_rockmigrations user@your.server.com

Create the workflow yaml


Now create the following yaml file in your repo:

# code .github/workflows/deploy.yaml
name: Deploy via RockMigrations

# Specify when this workflow will run.
# Change the branch according to your setup!
# The example will run on all pushes to main and dev branch.
on:
  push:
    branches:
      - main
      - dev

jobs:
  test-ssh:
    uses: baumrock/RockMigrations/.github/workflows/test-ssh.yaml@main
    with:
      SSH_HOST: your.server.com
      SSH_USER: youruser
    secrets:
      SSH_KEY: ${{ secrets.SSH_KEY }}
      KNOWN_HOSTS: ${{ secrets.KNOWN_HOSTS }}

Commit the change and push to your repo. You should see the workflow showing up in Github's Actions tab:

img

Once you got your SSH connection up and running you can setup the deployment. Remove or comment the job "test" and uncomment or add the job "deploy" to your deploy.yaml:

jobs:
  deploy:
    uses: baumrock/RockMigrations/.github/workflows/deploy.yaml@main
    with:
      # specify paths for deployment as JSON
      # syntax: branch => path
      # use paths without trailing slash!
      PATHS: '{
        "main": "/path/to/your/production/webroot",
        "dev": "/path/to/your/staging/webroot",
      }'
      SSH_HOST: your.server.com
      SSH_USER: youruser
    secrets:
      SSH_KEY: ${{ secrets.SSH_KEY }}
      KNOWN_HOSTS: ${{ secrets.KNOWN_HOSTS }}

If you are using submodules just set the SUBMODULES input variable and add a CI_TOKEN to your repo secrets:

# .github/workflows/deploy.yaml
name: Deploy via RockMigrations
on:
  push:
    branches:
      - main
      - dev
jobs:
  deploy:
    uses: baumrock/RockMigrations/.github/workflows/deploy.yaml@main
    with:
      # specify paths for deployment as JSON
      # syntax: branch => path
      # use paths without trailing slash!
      PATHS: '{
        "main": "/path/to/your/production/webroot",
        "dev": "/path/to/your/staging/webroot",
      }'
      SSH_HOST: your.server.com
      SSH_USER: youruser
      SUBMODULES: true
    secrets:
      CI_TOKEN: ${{ secrets.CI_TOKEN }}
      SSH_KEY: ${{ secrets.SSH_KEY }}
      KNOWN_HOSTS: ${{ secrets.KNOWN_HOSTS }}

See https://bit.ly/3ru8a7e how to setup a Personal Access Token for Github. You need to create this token only once for your Github Account, not for every project, but you need to add it to every project that should be able to access private submodules!

Your workflow should copy files but fail at step Trigger RockMigrations Deployment. That is because you need to create a site/deploy.php file:

// code site/deploy.php
<?php namespace RockMigrations;
require_once __DIR__."/modules/RockMigrations/Deployment.php";
$deploy = new Deployment($argv, "/path/to/your/deployments");
// custom settings go here
$deploy->run();

Note that you must set a path as second argument when creating a new instance of Deployment. This path ensures that if you run your deployment script on another machine (for example on a local DDEV environment) it will run "dry" and will not execute any commands. This only works if your local path is different from your remote path of course!

This is how it looks like if everything worked well:

img

Debugging


Debugging can be hard when using CI/CD pipelines. If you get unexpected results during the PHP deployment you can make the script more verbose like this:

...
$deploy->verbose();
$deploy->run();

Migration Examples


Field migrations

CKEditor field

$rm->migrate([
  'fields' => [
    'yourckefield' => [
      'type' => 'textarea',
      'tags' => 'MyTags',
      'inputfieldClass' => 'InputfieldCKEditor',
      'contentType' => FieldtypeTextarea::contentTypeHTML,
      'rows' => 5,
      'formatTags' => "h2;p;",
      'contentsCss' => "/site/templates/main.css?m=".time(),
      'stylesSet' => "mystyles:/site/templates/mystyles.js",
      'toggles' => [
        InputfieldCKEditor::toggleCleanDIV, // convert <div> to <p>
        InputfieldCKEditor::toggleCleanP, // remove empty paragraphs
        InputfieldCKEditor::toggleCleanNBSP, // remove &nbsp;
      ],
    ],
  ],
]);

Image field

$rm->migrate([
  'fields' => [
    'yourimagefield' => [
      'type' => 'image',
      'tags' => 'YourTags',
      'maxFiles' => 0,
      'descriptionRows' => 1,
      'extensions' => "jpg jpeg gif png svg",
      'okExtensions' => ['svg'],
      'icon' => 'picture-o',
      'outputFormat' => FieldtypeFile::outputFormatSingle,
      'maxSize' => 3, // max 3 megapixels
    ],
  ],
]);

Files field

$rm->migrate([
  'fields' => [
    'yourfilefield' => [
      'type' => 'file',
      'tags' => 'YourTags',
      'maxFiles' => 1,
      'descriptionRows' => 0,
      'extensions' => "pdf",
      'icon' => 'file-o',
      'outputFormat' => FieldtypeFile::outputFormatSingle,
    ],
  ],
]);

Options field

$rm->migrate([
  'fields' => [
    'yourfield' => [
      'type' => 'options',
      'tags' => 'YourTags',
      'label' => 'Options example',
      'options' => [
        1 => 'ONE|This is option one',
        2 => 'TWO',
        3 => 'THREE',
      ],
    ],
  ],
]);

Options field with multilang labels:

$rm->createField('demo_field', 'options', [
  'label' => 'Test Field',
  'label1020' => 'Test Feld',
  'type' => 'options',
  'optionsLang' => [
    'default' => [
      1 => 'VERYLOW|Very Low',
      2 => 'LOW|Low',
      3 => 'MIDDLE|Middle',
      4 => 'HIGH|High',
      5 => 'VERYHIGH|Very High',
    ],
    'de' => [
      1 => 'VERYLOW|Sehr niedrig',
      2 => 'LOW|Niedrig',
      3 => 'MIDDLE|Mittel',
      4 => 'HIGH|Hoch',
      5 => 'VERYHIGH|Sehr hoch',
    ],
  ],
]);

Note that RockMigrations uses a slightly different syntax than when populating the options via GUI. RockMigrations makes sure that all options use the values of the default language and only set the label (title) of the options.

Page Reference field

$rm->migrate([
  'fields' => [
    'yourfield' => [
      'type' => 'page',
      'label' => __('Select a page'),
      'tags' => 'YourModule',
      'derefAsPage' => FieldtypePage::derefAsPageArray,
      'inputfield' => 'InputfieldSelect',
      'findPagesSelector' => 'foo=bar',
      'labelFieldName' => 'title',
    ],
  ],
]);

Date field

$rm->migrate([
  'fields' => [
    'yourfield' => [
      'type' => 'datetime',
      'label' => __('Enter date'),
      'tags' => 'YourModule',
      'dateInputFormat' => 'j.n.y',
      'datepicker' => InputfieldDatetime::datepickerFocus,
      'defaultToday' => 1,
    ],
  ],
]);

Install and use modules at your own risk. Always have a site and database backup before installing new modules.

Twitter updates

  • Introduction to an invoice application profile being built in ProcessWire: More
    30 September 2022
  • Stumbling upon a really nice ProcessWire-powered website, plus core updates including API improvements for ProcessWire forms— More
    9 September 2022
  • Useful new dot-and-bracket syntax options added for page.get() method— More
    2 September 2022

Latest news

  • ProcessWire Weekly #438
    In the 348th issue of ProcessWire Weekly we're going to cover the latest weekly update from Ryan, take a closer look at a couple of new third party modules, and more. Read on!
    Weekly.pw / 1 October 2022
  • Multi-language field translation export/import
    In this post we cover the details of a new module that enables export and import capabilities for multi-language fields in ProcessWire.
    Blog / 5 August 2022
  • Subscribe to weekly ProcessWire news

“…building with ProcessWire was a breeze, I really love all the flexibility the system provides. I can’t imagine using any other CMS in the future.” —Thomas Aull