Jump to content

How to sync databases?


Recommended Posts

Hello guys,

I'm trying to figure out how to sync fields and templates between staging and production environments.

I've found Migrations module by Lostkobrakai, but with use of it all the fields and templates must be created by API, which is kind of uncomfortable.

I also tried ProcessDatabaseBackups module which can export only certain tables, but I don't think it's the best practice to do that.

How do you guys solve this problem? It's very annoying to setup everything three times (dev, staging, production).

Thanks a lot :)

Link to comment
Share on other sites

Yeah, coming from several applications written from scratch, with codeigniter, and to a lesser extent, wordpress, workflow with Processwire has been tough for me personally, but I am adapting.

I've sort of settled into a pattern where after having deployed the initial production application, I try to do as much non-destructive work as possible on this instance.

If something may be destructive, then I'll clone the codebase and hit the same database and work on this. If and when everything looks good, I'll just point the DNS to this instance. You can read up on multi-instancing.

The reason for the above approach is because with the app I'm working on, the DB is too big to be slinging back and forth to a development machine.

Having said all this, while I'm working on production and staging instances, I've also got a half dozen R&D instances on my dev machine than are pretty darn close in structure to production.

This all probably sounds nebulous and I hope it helps. And I would certainly keep search-engine'ing things like "processwire development staging production"

  • Like 1
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Similar Content

    • By LuisM
      A companion Module for RockMigrations to gather and execute Migrations in one place. !Alpha RELEASE!
      Hi there,
      I just wanted to share my take on Migrations and how I am using RockMigrations to make my deployments somehow manageable. 
      First of all, a big shoutout to @bernhard for his efforts and accomplishments for our beloved CMS. 
      We recently had a video-call and talked about our daily problems and how to handle them, with a focus on Migrations. As of now you might think 'Gosh, not another Migrationsthingy' but wait. 
      Wouldnt it be nice to have a nice overview or some kind of list regarding Migrations and stuff and one place to manage them?
      Speaking for my own, yes I would like to have something like this. So yeah, here we are with a Companion Module for RockMigrations. 
      Product Requirements
      I want to manage Migrations on the Application level (done) I want to have one place in my FileSystem to place my Migrations (done) I want to see in which state my Migrations are (done) I want to have an execution order, execute migrations one by one (done) I want to trigger Migrations via CLI (open) I want to group multiple Migrations into One Migration to create Build Versions (open) I want to Rollback Migrations (open) I want to create a deliverable build out of migrations (open) I want to track changes to templates and fields and create migrations accordingly (open) I want to manage Migrations on the Module level (open) Module Requirements
      ProcessWire 3.0.178 RockMigrations latest PHP 7.4 Composer Current release: v1.0.2Alpha
      How does it work?
      On installation the Module will create a new Database Table, here we will hold information about migrations states.
      Your Migrations are empty on first installation, so just click on 

      You can now choose which Type of Migration you want to create, be it Templates, Fields or Pages and a according Action. 
      Im still working on this function, so yeah its a lil bit confusing right now. 
      The philosophy here is, every Type of Migration could be easily identified and we can Migrate on a very sepcific and granular base. As you can see in my Screenshot, I am using migrations to build an entire App from Migrations.
      After creating the new Migration File, switch over to your IDE and find a timestamped .php file inside modules/FlowtiCore/migrations. 
      This file just returns a simple Array.

      This is all the Info RockMigrations needs to do his thing while we keep track of the execution. Easy.
      Values Arrays have to follow PW and RockMigration field naming Conventions. 
      To make a Migration Executable set 'boilerplate' to false.
      After creation of your first Migrations your overview should be filled so lets go. We have to migrate our files in a specific order just to make sure we can create page_references and Parent/Child connections. To achieve this, Migrations are timestamped on creation. The older the higher the priority. To enforce execution order, every migration needs to have his predecessor executed and installed.

      How could this help my workflow?
      My workflow is based on Git, Webhooks and Git Events. So whenever I merge 'staging' into 'master', a build will be created and a deliverable should be pushed to a Server via SSH. 
      The Problem with ProcessWire is the lack of support for such Workflows and Toolchains due to its User-Friendly Admin Backend which is fine for a simple website but not suitable long-term if working in a multi-tenant environment or with more developers in a dev-staging-test-production setup.
      My Goal is to provide methods and ideas to support such workflows but also support a User-Friendly Interface to work with migrations.
      I really hope it could be of use for someone.
      I will add the Module to the Directoy once it reached a stable state.
      But you can get current version at GitHub
      Just clone it to your Modules Directory.
      The Module will create a new Database-Table on creation. The Default Name will be 'flowti_core_migrations'. To change this just edit 
      const DATABASE_TABLE = 'flowti_core_migrations'; Inside the Module Class. 
      Thats it, from there on just create new Migration Files, edit them and execute them.

    • By jom
      Chäs & Co is a b2b cheese trader. They love good cheese and are proud of their excellent knowledge und connections to small cheese producers. They maintain a custom made database and used to copy/paste (!) the cheese infos to the former CMS (Contao). Since substituting the database with ProcessWire was no option, they export now the database to a csv file and import it to the new website (expanded version of module ImportPagesCSV).
      From there, the pw magic begins. The staff works with the admin interface: They search and find products in notime using the build-in search field (using cheese names or id. Using ids only possible by adding hook, see this forum thread). They generate price lists easily based on a template, providing checkboxes and option buttons for some restrictions, choose i.e. the cheese origin and price audience (all implemented as page selectors). By saving the page a PDF list gets generated (using mpdf, a php pdf library).
      The visitors can sort and filter the products using a form by shuffle.js. Product details are shown in a popup (fancybox). There's also the option for generating a nice pdf data sheet. No CSS framework being used.
      Other modules used: ProcessJumpLinks, ProcessProMailer, ProcessWireUpgrade.
      Visual design: Nicole Haller

    • By Juergen
      Hello @ all,
      I am creating a new inputfield/fieldtype to store opening hours, but I am struggeling to save values from multiple dynamic created inputfields in 1 column of the database.
      The user can enter one or more opening times per day in a UI.
      Monday open from 08:00 to 12:00 and from 14:00 to 17:00 Tuesday open from 08:00 to 12:00 and from 14:00 to 19:00 and so on
      Via a little JavaScript you can add as much opening times as you need per day - the additional inputfield will be created dynamically.
      After form submission all the values are in the POST array -> this works (see example below):
      ProcessWire\WireInputData Object ( [openinghours_mo-0-start] => 09:00 [openinghours_mo-0-finish] => 13:00 [openinghours_mo-1-start] => 14:00 [openinghours_mo-1-finish] => 18:00 [openinghours_mo-2-start] => 21:00 [openinghours_mo-2-finish] => 23:00 [openinghours_tu-0-start] => 09:00 [openinghours_tu-0-finish] => 13:00 [openinghours_tu-1-start] => 14:00 [openinghours_tu-1-finish] => 18:00 [openinghours_we-0-start] => 09:00 [openinghours_we-0-finish] => 13:00 [openinghours_we-1-start] => 14:00 [openinghours_we-1-finish] => 18:00 [openinghours_th-0-start] => 09:00 [openinghours_th-0-finish] => 13:00 [openinghours_th-1-start] => 14:00 [openinghours_th-1-finish] => 18:00 [openinghours_fr-0-start] => 09:00 [openinghours_fr-0-finish] => 13:00 [openinghours_fr-1-start] => 14:00 [openinghours_fr-1-finish] => 18:00 [openinghours_sa-0-start] => [openinghours_sa-0-finish] => [openinghours_so-0-start] => [openinghours_so-0-finish] => ) The property name is always the name attribute of the field 😉 . If the property is empty means closed on that day.
      Now I need to combine all those values into 1 array (or json array) and store it in the database in 1 column called 'hours' in my case (see screenshot below):

      In my ___processInput(WireInputData $input) method I have tried to make it work like this:
      public function ___processInput(WireInputData $input): self { $name = $this->attr('name'); $value = $this->attr('value'); //input object includes always every input on the page, so lets filter out only inputs from this field //we need to do this, because the number of values is variable - so extract only values that starts with $name.'_' $nameAttributes = []; foreach($input as $key=>$value){ if(substr($key, 0, strlen($name.'_')) === $name.'_'){ $nameAttributes[$key] = $value; } } // loop through all inputfields of this fieldtype $time_values = []; foreach($nameAttributes as $nameAttr => $value) { $time_values[$nameAttr] = $value; } } //save it in the database $input->set('hours', serialize($time_values)); return $this; } The only important part of this code is the last part with the serialize function.
      After saving it will create a record in the database, but the value is always NULL (default value) (see below).

      Checking $time_values returns all the values, but printing out "$this" shows me that the property "hours" inside the Openinghours object is empty (see below) - so the mistake must be there, but I dont know where?!?!?!?
      [title] => Home [openinghours] => ProcessWire\OpeningHours Object ( [data] => Array ( [hours] => ) ) If I check the sleepValue() method or the sanitizeValue() - they are also empty. So it seems that the values will not reach these methods. I havent found a clear documentation of whats going on behind the saving process of an inputfield.
      As far as I know the saving process starts with the form submission. The values are in the POST array and will be processed by the processInput() method. Before they will be saved in the database they will be sanitized by the sanitizeValue() mehtod and afterwards they will be prepared for storage in the sleepValue() method.  The last step is the storage itself.
      Has someone an idea what is missing by storing values from multiple fields into 1 database column or has someone a working example of such a scenario on github to help me out.
      A clear explanation of the storage process will be also helpful.
      Thanks and best regards
    • By Pip
      Hi Everyone 
      I've been working on Processwire for two months now. Structuring the website as needed. Unsure why but I'm getting this one now. Seems my fields has crashed. 
      I've tried googling some answers but can't seem to find a step by step guide on how to rectify this. Any advise? Practically new on this. 

    • By Rodd
      Hi everyone!
      I have a website in a production environment and I want to duplicate it in a local environment. I exported the content of the website (with the 'Site Profile Exporter' module) but I cannot use it actually. I've got an issue with the database. I imported this one in MAMP then.

      I also exported the pages (with the 'ProcessPagesExportImport' module), but I cannot import it to my local website because the fields don't exist. So I created this fields, but I have this error :
      How can I use the elements that already exist and are presents in my database? How can I duplicate correctly the templates, fields and pages?
      Thanks by advance
      PS: Sorry if my english is bad
  • Create New...