Using Config Migrations with an existing project

[Kiwi Chris]

I love the concept of config migrations with each field and template having its own file, as it's much more in line with what I'm used to with ASP.Net Core and Visual Studio where you can build some things visually, but still end up with code, and it's also tidier with version control, as fields or templates that don't change are kept separate.

Another benefit I see of using config migrations, is that due to module dependencies, unlike site profiles that are all or nothing, it's possible to define a basic set of fields for a site in a module, even if the module doesn't do much other than install fields and templates, and then you have a working site with some base functionality.

If you want additional functionality on some sites but not others, but require certain fields or templates to exist before you add the additional functionality, the module dependency can take care of this.

eg, I might start off wanting just a basic site, then want to add a blog, but a blog template will use some of the same fields as the basic site, but I don't want to have to redefine them, just be sure they already exist before I install the blog upgrade.

The issue I've struck is an existing project that I've defined fields and templates with using the ProcessWire UI, but now I want to bundle some of them up into a module.

The scenario is I've started building what I think is a one-off site, then I realise that different components have the potential to be reused on multiple sites, but exporting a site profile is too inflexible.

Here's a screenshot of templates after copying and pasting template data from existing templates into config migrations for two new modules and installing them.

As you can see there are duplicate templates, the originals with no prefix, and new ones prefixed with the module name.

I want to avoid this, but still like using the ProcessWire UI to design fields and templates, because I'm forgetful and can't remember all the properties and what they do, and I find the UI reminds me, but once I've created or updated a field or template, I want to add it to a migration file so that it's easy to update across multiple sites.

Would renaming the templates (same thing applies to fields) to the naming convention of config migrations avoid this doubling up?

Also, if I edit one of these apparent duplicate fields, I get this:

If I edit the actual "animal" template I get all the usual field editing options, with the normal warning that RockMigrations is installed, although in my config file I've got:

$config->noMigrate = true;

to prevent rockMigrations overriding changes I make via the ProcessWire UI on my development site, although installing the modules still appears to have triggered the config migrations.

Ideally I'd like to avoid what looks like two definitions of the same thing, and still have the UI based editing available.

What's the best way to achieve this?

 

[bernhard]

17 hours ago, Kiwi Chris said:

I love the concept of config migrations with each field and template having its own file

me too 😉 

17 hours ago, Kiwi Chris said:

I want to avoid this, but still like using the ProcessWire UI to design fields and templates, because I'm forgetful and can't remember all the properties and what they do, and I find the UI reminds me, but once I've created or updated a field or template, I want to add it to a migration file so that it's easy to update across multiple sites.

I'd love to have a feature, where you add/setup fields via the GUI and then click a button "save as migration file" with all the changed properties that differ from the default. But unfortunately it's not so easy (because some properties have slightly different syntax or you might get circular references or such things).

And we have snippets, which makes it very easy to create fields. Once they are in place (when using VSCode/Cursor/...) you create a new file with the name of the field/template/etc.. and you type "rmf-text" and you'll get the code for a textfield migration. Then you chance all the settings that you need (or do it via the GUI and copy things over) and that's it. You have to remember maybe 10 properties/words to get 90% of the migrations that you need (columnWidth, label, type, notes, collapsed).

--------

Not sure what is going on in your screenshot. Looks like it's using the fieldgroup of another template. Don't know why that's the case but it's nothing that I see in any of my projects.

Regarding the prefix: Not sure if your migrations need to be part of a module? You can also place them in site/RockMigrations and there a file whatever.php will create a field called "whatever", not "someprefix_whatever". Then you can just copy and paste the templates and fields that you need over to the next project. But as soon as you want to make things more isolated and want to have migrations in a module the prefix makes sense as it will make sure we don't get naming collisions (eg multiple "body" fields or such).

Hope that helps.

[Kiwi Chris]

1 hour ago, bernhard said:

I'd love to have a feature, where you add/setup fields via the GUI and then click a button "save as migration file" with all the changed properties that differ from the default.

This would be the ultimate development tool. I understand how it could be difficult to implement, as I found out the hard way with repeaters that if you save all field properties into a migration file it deletes data when applied. So far I haven't experienced anything so catastrophic with any other field types.

I wonder if in a migration for a module at least, that's using config migrations, whether it would be possible to define a watch list of fields and templates so that any time these are changed via the GUI, their individual migration file will be updated? They probably would still need some manual editing to remove unnecessary properties, but a good diff tool can do this quickly.

1 hour ago, bernhard said:

And we have snippets, which makes it very easy to create fields.

I hadn't spotted that before. Very nice feature to avoid having to remember lots of stuff. I think I can remember 'rmf'. 🙂

1 hour ago, bernhard said:

Not sure what is going on in your screenshot. Looks like it's using the fieldgroup of another template.

What happened here is I defined the fields via the UI as normal fields without any prefix, then copied the definitions from the UI into config migration files.

When I installed the module, the fields got created, but with prefixes. 

I will experiment with renaming the fields and templates in the GUI before I add them to config migrations, and see if this gives me just a single copy of each field and template.

1 hour ago, bernhard said:

Regarding the prefix: Not sure if your migrations need to be part of a module? You can also place them in site/RockMigrations and there a file whatever.php will create a field called "whatever", not "someprefix_whatever".

Do I still need to follow the convention of site/RockMigrations/templates, site/RockMigrations/fields etc?

I made the files in a module because I thought config migrations require a module, but if they work at the site level as well, that might be more appropriate.

My migrations probably don't need to be part of a module however I started thinking about a website itself like an object. Even the most basic website will probably have certain fields and templates to be functional. Depending on what else it does, it will need additional fields and templates that extend that base. If I decide I want to refine that base, I can do it in one place and apply it to all sites that use it without having to copy and paste any code if I have it defined in a module.

 

 

[bernhard]

16 hours ago, Kiwi Chris said:

I wonder if in a migration for a module at least, that's using config migrations, whether it would be possible to define a watch list of fields and templates so that any time these are changed via the GUI, their individual migration file will be updated? They probably would still need some manual editing to remove unnecessary properties, but a good diff tool can do this quickly.

If you want to implement this go ahead. I'm not going to do that, because it's a free module and I don't have any benefit in doing so, because I'm faster writing migrations by hand. It's very fast and it's very reliable. Any other steps involved add possible errors and there might be several edge cases to think of. But I guess this would be great for some people, that's why I said I'd love to have it work like this.

From what I thought about it so far I thought it would make sense to have some kind of "check-in" where you click on "watch this field for changes" and then it stores the current config migration somewhere and only writes changed properties to the config migration file. But again - it's nothing that I need for myself.

16 hours ago, Kiwi Chris said:

What happened here is I defined the fields via the UI as normal fields without any prefix, then copied the definitions from the UI into config migration files.

I guess you have some hardcoded ids in your config migrations. That's another problem with these automated exports compared to writing migrations on your own. If you write a migration on your own you can use template/field/etc NAMES instead of ids. That leads to a lot better readable migrations (field => "whatever" vs. field => 123). I guess you have a fieldgroup_id => 123 somewhere and that's the same id of the fieldgroup that your original template uses. In PW in the GUI you only define the template, but internally a template has a fieldgroup that defines the fields that the template uses. And multiple templates can use the same fieldgroup and this is what you are seeing.

16 hours ago, Kiwi Chris said:

Do I still need to follow the convention of site/RockMigrations/templates, site/RockMigrations/fields etc?

Please check out the docs. If you are missing something let me know.

16 hours ago, Kiwi Chris said:

My migrations probably don't need to be part of a module however I started thinking about a website itself like an object. Even the most basic website will probably have certain fields and templates to be functional. Depending on what else it does, it will need additional fields and templates that extend that base. If I decide I want to refine that base, I can do it in one place and apply it to all sites that use it without having to copy and paste any code if I have it defined in a module.

Yeah I have always tried to have some kind of starter profile / module that I can reuse, but in reality most of the time I start over from scratch as it's fast and you don't get any overhead... But as soon as you have a feature that makes sense for more projects just create a module for it and migrations will be your best friend.