Merge databases

[tkam]

Hi!

I have a shop site running on processwire, which I just finished developing. 
Now there is the challenge of developing more features, while the shop is already in use and running. When I start a new feature, I copy the live database to my dev server and start developing with the most recent data. But in the meantime there'll be new users and orders on the live system.

What is the best approach of merging both databases? Is there a good helper tool or is the only way to write a migration script?

And if the migration script is the best way - how can I connect from my dev system to the local and live database at the same time?

Cheers, Tom

[elabx]

I think the current best way out to handle this is using RockMigrations. And yes, you write a migration script that runs on the live DB, to updated whatever configuration you need on processwire templates/fields. Be sure to have your backups in place ?

[AndZyk]

ProcessWire also offers build in export and import functions for templates, fields and pages. ?

[elabx]

2 hours ago, AndZyk said:

ProcessWire also offers build in export and import functions for templates, fields and pages. ?

I can confirm I use this a lot too!! It also merges updates on fields and templates. I think the only caveat I have is that it doesn't work 100% with FieldtypeOptions.

And the pages import/export is crazy good, works with absolutely every native field I've tested with, even complex Repeater Matrix/Repeaters. I just recently used it to migrate some old sites, into a new "template field schema" and I cannot even estimate how much work it saved lol

[bernhard]

RockMigrations was built for exactly this purpose and has been in development for 3 years now and I'm using it on a daily basis for all my projects.

The new version is already online, ready to use and so much better and easier than the old version ?  I just have not released it yet because I plan to do a video introduction for it so that it is easy for everybody to see how it works and how easy it is to use. https://github.com/baumrock/rockmigrations

The new version has a helper for github actions for fully automated deployments. So when set up correctly all you have to do is "git push" and everything on the live server will be updated.

When working locally all you have to do is to restore the live database. RockMigrations has a files-on-demand-feature that will automatically download any missing assets from your live server: https://github.com/baumrock/rockmigrations#file-on-demand

[tkam]

Thanks guys, I knew about the import / export function, but was worried, that it would not work reliably when I have a lot of pages, like 100 or more. Did anyone test this with more than 20 pages?

I'll have a look at RockMigrations. Thanks for the quick input, you guys are awesome! ? 

[tkam]

Ok, I just had a look at RockMigrations. This helps me to create new fields and templates, but what about the pages and content? 

Currently I am working locally on the site translation, so I've got tons of new content all over the place. In the meantime there'll be new content on the live site.

Would the best way then to work with the import / export page function? And is there a limit to the amount of pages I can import or export?

[bernhard]

1 hour ago, tkam said:

Currently I am working locally on the site translation, so I've got tons of new content all over the place.

Translations are a topic on its own, especially with migrations...

If you need to create Pages, in RockMigrations you can do this:

$rm->createPage('Your page title', 'your-page-name', #parentpage, ['hidden', 'locked'])

Which will create the page only if it does not exist.

This might be what you need or not. You can create pages very easily via the PW API... Or use import/export tools. Don't know if there are limits regarding the amount of pages.