thlinna

Best practise to implement changes in PW from development to production

Recommended Posts

I am running development server where I test and try the changes in fields and templates of ProcessWire. The php files are stored in git@bitbucket. 

The part of managing changes and versions of php code itself via git is quite straightforward, as I presume we all know. However, I have not been able to come up with decent way of moving the changes in fields, templates, modules and settings from development to production other than repeating them, step-by-step, manually in production environment via the admin back-end.

Perhaps there is more sophisticated way of transferring these changes?

Share this post


Link to post
Share on other sites

Hi @thlinna this has come up a couple of times, here's a selection of topics, perhaps you can find something useful in there.

Tip: use google search for searching the forums every now and then, I find it spits out more useful results at times:

search: sync live dev site:processwire.com/talk

 

Share this post


Link to post
Share on other sites

Thanks you so much for the insightful replies, @owzim and @AndZyk. Of course I tried to find something with PW forum search, but was not happy with the results I found.

For my production site has active content creation and new content is stored in database all the time. My development environment is a snapshot of certain time in production, so it's content start getting outdated from the second I take the snapshot. 

I looked promptly at @LostKobrakai's Migrator-module, but the use of it felt quite complicated, especially for newbie and for beta status. I hope no-one would not take this statement offending, but I feel sometimes tutorials and forum thread information require more indepth understanding of PW itself. As a newbie one is still trying to overcome the steepness of the learning curve. If one hits a wall, it may take plenty of time to overcome it. PHP environment is quite different from java or c#, so my coding and setup skills for the latest devops style coding is shallow.

Maintaining PHP code, CSS and JS is a OK with Git, but how to move created pages from dev to prod is not that simple anymore. As @Michael Murphy pointed out, in another thread, one way is by exporting/importing fields and templates. But does it export the pages or content that I have created in the dev environment in to the fields?

Share this post


Link to post
Share on other sites

@thlinna I have been working on a module, a more non-dev-friendly approach the migrations module for about one year (on/off). I am actually using it all the time but it's not ready for public. It has many bugs I know how to circumvent :) and next to no documentation. It's similar to JSON Installer module, but more powerful and the data files are written in YAML.

Perhaps JSON Installer can be of help to you.

  • Like 3

Share this post


Link to post
Share on other sites

@thlinna I do not know if it will be of any help, but I myself do not migrate any db at all. Instead I build my workflow around this: http://tltech.com/info/mysql-via-ssh/

I put together a bash script to completely automate the process, and normally I can clone the live db to my local machine in seconds (small databases...). This way I always have to work on the live site to make all the necessary changes, afterwards I have to clone the db, but it is an easy to follow workflow and dead simple :) It might be too simple for teams, but a solo developer cannot go wrong with it, at least I'm quite happy with it, I got used to working this way. Sure, I have never had to work with really big databases, but these days even transferring dozens of megabytes does not take long.

  • Like 3

Share this post


Link to post
Share on other sites

@szabesz sometimes live changes break the existing site if you don't have the necessary files (e.g. template files). Also, working on a live site's structure ... breaking stuff. Rather locally.

It's kind of cumbersome, but this is how I do it: I lock the site for edits (in consultation with the client of course), clone the db, make changes locally and update the db when finished.

Here's a small recipe for adding a maintenance mode.

  • Like 1

Share this post


Link to post
Share on other sites

Yep, true, of course, it is not always just about adding a field here and there... So while my "migrating workflow" is simple, it comes with the issues you mention, so yes, sometimes I do have to try things out on my machine before I redo it on the live site, but this does not happen every day. And also true that "maintenance mode" with kindly locked out users might also be needed sometimes.

To get around the problem of missing files, I use Beyond Compare, the fastest an easy to use diff tool out there. When it is set to e.g. ftp 8 simultaneous connections, it can deal with a site in 10-30 seconds or a few minutes, depending on the number of files. I only use git to manage my own code.

EDIT: My bash script can also migrate/clone complete sites in a minute or two, just in case a need another "branch" of the whole site to work on something that takes long and should not interfere with my regular mirrored version of the live site.

Edited by szabesz
typos

Share this post


Link to post
Share on other sites

Right now, I use rsync for code deployement and about to install a bash script running mysqldump and mysql for DB backup and deployment. I run Procache so in the midst of writing some php modules that reconfigure the site from Dev to Prod.

Hopefully I'll have a script that ties everything together soon.

If this is of interest to anyone, I could post it once I'm sufficiently happy with it?

 

 

  • Like 2

Share this post


Link to post
Share on other sites
13 minutes ago, FrancisChung said:

If this is of interest to anyone, I could post it once I'm sufficiently happy with it?

I'm interested :) Thanks is advance.

Share this post


Link to post
Share on other sites
1 hour ago, FrancisChung said:

If this is of interest to anyone, I could post it once I'm sufficiently happy with it?

Interested too!

Share this post


Link to post
Share on other sites
19 hours ago, thlinna said:

..., but the use of it felt quite complicated, especially for newbie and for beta status. I hope no-one would not take this statement offending, but I feel sometimes tutorials and forum thread information require more indepth understanding of PW itself.

That's mostly because it's not a simple problem to solve at all. Merging of changes happening in two independent systems cannot work simply by clicking a button. There are various ways to make it work, which all come with their own kind of backdraws. With each project having different goals/needs it's a matter if choosing the one giving you the least pains, while helping you enough to make it worth. 

Also I'm not sure if I missed some beta statement in my modules topic, but it's really stable. Any upcoming changes should only affect the accompanied cli tool. Also I've it in production usage for over a year now with no issues in the latest months.

  • Like 5

Share this post


Link to post
Share on other sites
6 minutes ago, LostKobrakai said:

Also I'm not sure if I missed some beta statement in my modules topic, but it's really stable.

You might want to remove the beta status. No software is perfect, however if it is stable, why not give it 1.0.0? :) 

  • Like 2

Share this post


Link to post
Share on other sites
4 hours ago, LostKobrakai said:

Also I'm not sure if I missed some beta statement in my modules topic, but it's really stable. Any upcoming changes should only affect the accompanied cli tool. Also I've it in production usage for over a year now with no issues in the latest months.

Oops. My bad, I apologize. My assumption was based on some old forum threads. 

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By louisstephens
      I have been following @bernhard's great tutorial on creating a Dashboard and everything has very easy to understand so far. However, I need to add a select field that a template is using (so users can create a new page from the dashboard), but I am really at a complete lost on how to achieve this. I have been googling, but can't really find anything on the matter.
      $field = $this->modules->get('InputfieldSelect'); $field->name = "Select Manufacturer"; $field->findPagesSelector = "template=basic-page"; //not needed for select field $field->labelFieldName = "title"; $field->name = "Manufacturer Select"; $field->columnWidth = 34; $fieldset->add($field); I have the above code, which was originally an InputfieldPage, but since this is an options field, I have changed it. Has anyone actually gotten a select from a template (and its' values) in a module?
    • By rickm
      Hey all,
      A while back I was trying to get Processwire working on the Homestead vagrant box - a really handy vagrant box made by Taylor Otwell (Laravel creator). One cool thing about homestead is it has its own config file where you can set 'path maps' which map a folder on your computer to the vagrant box, and then also set up hostnames. So for example you can set up 10 domains pointing to different locations on your machine, making it a nice all in one vagrant development box for your sites. 
      One thing it didn't do was support processwire. The way it works is it uses setup files for each site, and you can specify things like a site to use nginx, a site to use apache, etc. 
      So to get round this I wrote a custom setup script for it to get processwire working, and it works really well. I've put it up on github should anyone else find it handy, it's a pretty simple little script and is based on the Laravel one included in Homestead.
      Hopefully it'll be useful to someone out there  You can grab it here: https://github.com/rickmills/homestead-processwire
    • By prestoav
      Hi folks,
      I have a marketplace site built on PW which I'd like to tie to a subscription payment system for those selling. The user accounts currently use specific PW user accounts. I'm looking closely at PayWhirl to manage payments and subscriptions as it looks ideal for subscriptions. Their API seems to work well too from initial testing.
      I wondered if anyone else in this super forum had used PayWhirl and had any advice or tips & tricks?
      Thanks.
      https://app.paywhirl.com/
    • By John W.
      Summary: If I have a site sitting on a server called testing.com, then on the server change it to mycompany.com, do I have to modify any settings in ProcessWire?
      After building several processwire for clients, the one question that I haven't had is about moving a processwire site from an old (temporary) domain to a new domain.
      For instance, I typically setup development sites for clients and register a temporary domain (and lock with a password). For instance, thee client can go test.com and get all their content finalized, change wording, add news, swap out photos -- everything to make the site production read.  
      Next, on my hosting account I can have testing.com renamed to theircompany.com - and change the nameservers.
      A simple question here. If I have a hosting company rename the account from testing.com to theircompany.com, change my nameservers whereas going to theircompany.com loads the production site, Is there anything I need to change in the processwire configuration or in the PW database?
      In the past, I've just manually copied the site, and fresh-installed PW, and imported the data. Seems like a lot of hassle.
      I have a second part question about updating the PW core, however, after consideration I'll post it as a new topic.