Jump to content

Development/Staging of Processwire site


Bjorn
 Share

Recommended Posts

Before getting to my question: Sorry if this has been discussed already...

I am reasonably new to using Processwire, but think it's absolutely great! I have built my site using it and will be moving it to a production environment for use by various people. 

My question is how to go about the ongoing development of the site when I use a development/staging area and not loose data in the production environment when I have to update?

As part of my ongoing development, I will be editing template files, possibly adding modules, configuring modules and making changes in the database i.e. new fields, modifying fields etc. When I need to update the live server, inserting/updating the files on the filesystem is easy enough (will probably use Git) - but how to update the database when users have been making their own updates & content via the front-end? Obviously my dev-version of the database will not have this new content... Re-playing all the modifications on the production server (i.e. config of modules, updating fields etc.) is also not really an option since this is very error-prone and a tedious affair.

Can someone possibly help me here?

  • Like 1
Link to comment
Share on other sites

The system that's proven more effective for me is:

1 — do all the non destructive database changes on the live site (adding fields and adding them to templates, creating new templates and so on)

2 — import the database to the development server and do the templates changes

3 — pull the new files to the live site without touching the database

4 — do some cleaning on the live database if needed (remove fields, templates and so on)

Another tip, although this one doesn't really answer your question, just because I think it can be useful :)

One system that I already used effectively was to create another install of PW (a copy of the other) in the live server (on a subdirectory or subdomain) that connects to the same database, and develop all the changes on the template files right there, using real and up-to-date content. Then I simply replace the template folder by the new one.

  • Like 6
Link to comment
Share on other sites

I'm using simple migration scripts, where I update anything via the api. It's a bit verbose and not as nice as using the backend, but it keeps db changes in a manageable and documented fashion. Also after a while it's more copy paste for most tasks.

Link to comment
Share on other sites

I have the vague impression that database migrations are better supported in other types of frameworks/languages (Rails/Ruby)? Or is Processwire simply not intended to have this type of implementation? Or am I misinterpreting something here? I can only guess from my experience, that database sync is not trivial.

It just sparks my curiosity that currently there are two threads asking about this, and also have read about a lot of people rather using static CMS due the "databaseless" way these apps work.

Link to comment
Share on other sites

I have the vague impression that database migrations are better supported in other types of frameworks/languages (Rails/Ruby)? Or is Processwire simply not intended to have this type of implementation? Or am I misinterpreting something here? I can only guess from my experience, that database sync is not trivial.

It just sparks my curiosity that currently there are two threads asking about this, and also have read about a lot of people rather using static CMS due the "databaseless" way these apps work.

It's an old problem, and no, there isn't a simple solution. I think the most commonly adopted solution is still this one, back from 2011 http://ben.kulbertis.org/2011/10/synchronizing-a-mysql-database-with-git-and-git-hooks/

  • Like 1
Link to comment
Share on other sites

Database migrations are "better" supported in other frameworks because they wouldn't function without them. A laravel framework doesn't have any databases beside those setup by migrations. But these are actually really bare bone in that they mostly update schema changes. For migrating db rows most use database seeding libraries, which then create the necessary models to be stored. Both can be done with the processwire api without actually needing to learn something new. 

This is basically what model factories do in frameworks (besides the autofill functionality). 

$pages->add('basic-page', $pages->get("/url/"), 'profile', array(
  "someField" => "value",
  "anotherField" => 202
);

And this may be what migrations do.

// Create field
$f = new Field();
$f->name = 'text';
$f->type = 'FieldtypeTextarea';
…

$f->save();

// Add to template
$bpt = $templates->get('basic-page');
$bpt->fields->add($f);
$bpt->fields->save();
  • Like 3
Link to comment
Share on other sites

Thanks for the comments guys! What diogo wrote is pretty much what I am doing now - but that would imply that I lock down the live site whilst I am developing, which is not ideal.

I do like LostKobrakai's proposal which I never considered before. It would make it slightly lengthy to build the instructions and more so to test it (when it fails, you have to restore DB backups, correct the instructions and then try again).
 
Indeed it looks as if there is no silver bullet here. Pity.
Link to comment
Share on other sites

The idea of the process that I described is exactly to avoid locking the live site. That's why doing all the nondestructive changes on the live site and avoiding any DB changes on the local copy.

  • Like 1
Link to comment
Share on other sites

I'm using simple migration scripts, where I update anything via the api. It's a bit verbose and not as nice as using the backend, but it keeps db changes in a manageable and documented fashion. Also after a while it's more copy paste for most tasks

Can you show some examples of this ?

This would be handy to update only parts of the online database and leave the rest alone.

Let's say you develop local on your laptop and want to update the database of the live site on the host.

If you have a guestbook, pagecounter, subscriptions, etc on the live site then these will be overwritten

in the online database each time you update from your local dev site.

  • Like 1
Link to comment
Share on other sites

Before PW my approach was to keep data of the content and data about the system in separate tables (and files). That helped a lot.

Given PW's great API, migration scripts is the way to go. I'm not there yet but I've been doing massive data imports using scripts and API and am very pleased with how well that works.

The challenge is keeping track of what you did while trying out different approaches and refining things. Probably makes sense sometimes to have Sandbox, Staging, Live. Do the messy creative work in the sandbox. Reproduce those changes deliberately and systematically on a fresh copy (Staging) using your scripts. Repeat as needed. Then (make a backup and) apply scripts to Live.

  • Like 1
Link to comment
Share on other sites

@pwired

There's nothing fancy about it. I've a small process module as well, which does the execution of the functions as well as storing which migrations are migrated and which aren't. 

// Migration.php
abstract class Migration extends Wire{

	public static $description;

	abstract public function update();

	abstract public function downgrade();

}

// 2016-03-03_18-27.php

class Migration_2016_03_03_18_27 extends Migration {

	public static $description = "Update Homepage title";

	public function update() {
                $this->pages->get('/')->setAndSave('title', 'Super fancy new Homepage title');
	}

	public function downgrade() {
                $this->pages->get('/')->setAndSave('title', 'Home');
	}

}
  • Like 3
Link to comment
Share on other sites

Thank you for the snippet lostkobrakai. Do you think it would somehow be possible or make sense to have kind of an export button (like the one at templates) that makes it possible to copy/download a script with all the API commands to create an exact copy of that field with all settings?

  • Like 1
Link to comment
Share on other sites

@LostKobrakai, thanks for coming back on this.

I guess if one understands the api, mysql and php enough, you could select only specific database tables to update,

leave the rest of the database tables alone and run your own database updater.

Link to comment
Share on other sites

  • 1 month later...

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
 Share

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...