Jump to content

Best practise to implement changes in PW from development to production


Recommended Posts

I am running development server where I test and try the changes in fields and templates of ProcessWire. The php files are stored in git@bitbucket. 

The part of managing changes and versions of php code itself via git is quite straightforward, as I presume we all know. However, I have not been able to come up with decent way of moving the changes in fields, templates, modules and settings from development to production other than repeating them, step-by-step, manually in production environment via the admin back-end.

Perhaps there is more sophisticated way of transferring these changes?

Link to comment
Share on other sites

Hi @thlinna this has come up a couple of times, here's a selection of topics, perhaps you can find something useful in there.

Tip: use google search for searching the forums every now and then, I find it spits out more useful results at times:

search: sync live dev site:processwire.com/talk


Link to comment
Share on other sites

Thanks you so much for the insightful replies, @owzim and @AndZyk. Of course I tried to find something with PW forum search, but was not happy with the results I found.

For my production site has active content creation and new content is stored in database all the time. My development environment is a snapshot of certain time in production, so it's content start getting outdated from the second I take the snapshot. 

I looked promptly at @LostKobrakai's Migrator-module, but the use of it felt quite complicated, especially for newbie and for beta status. I hope no-one would not take this statement offending, but I feel sometimes tutorials and forum thread information require more indepth understanding of PW itself. As a newbie one is still trying to overcome the steepness of the learning curve. If one hits a wall, it may take plenty of time to overcome it. PHP environment is quite different from java or c#, so my coding and setup skills for the latest devops style coding is shallow.

Maintaining PHP code, CSS and JS is a OK with Git, but how to move created pages from dev to prod is not that simple anymore. As @Michael Murphy pointed out, in another thread, one way is by exporting/importing fields and templates. But does it export the pages or content that I have created in the dev environment in to the fields?

Link to comment
Share on other sites

@thlinna I have been working on a module, a more non-dev-friendly approach the migrations module for about one year (on/off). I am actually using it all the time but it's not ready for public. It has many bugs I know how to circumvent :) and next to no documentation. It's similar to JSON Installer module, but more powerful and the data files are written in YAML.

Perhaps JSON Installer can be of help to you.

  • Like 3
Link to comment
Share on other sites

@thlinna I do not know if it will be of any help, but I myself do not migrate any db at all. Instead I build my workflow around this: http://tltech.com/info/mysql-via-ssh/

I put together a bash script to completely automate the process, and normally I can clone the live db to my local machine in seconds (small databases...). This way I always have to work on the live site to make all the necessary changes, afterwards I have to clone the db, but it is an easy to follow workflow and dead simple :) It might be too simple for teams, but a solo developer cannot go wrong with it, at least I'm quite happy with it, I got used to working this way. Sure, I have never had to work with really big databases, but these days even transferring dozens of megabytes does not take long.

  • Like 3
Link to comment
Share on other sites

@szabesz sometimes live changes break the existing site if you don't have the necessary files (e.g. template files). Also, working on a live site's structure ... breaking stuff. Rather locally.

It's kind of cumbersome, but this is how I do it: I lock the site for edits (in consultation with the client of course), clone the db, make changes locally and update the db when finished.

Here's a small recipe for adding a maintenance mode.

  • Like 1
Link to comment
Share on other sites

Yep, true, of course, it is not always just about adding a field here and there... So while my "migrating workflow" is simple, it comes with the issues you mention, so yes, sometimes I do have to try things out on my machine before I redo it on the live site, but this does not happen every day. And also true that "maintenance mode" with kindly locked out users might also be needed sometimes.

To get around the problem of missing files, I use Beyond Compare, the fastest an easy to use diff tool out there. When it is set to e.g. ftp 8 simultaneous connections, it can deal with a site in 10-30 seconds or a few minutes, depending on the number of files. I only use git to manage my own code.

EDIT: My bash script can also migrate/clone complete sites in a minute or two, just in case a need another "branch" of the whole site to work on something that takes long and should not interfere with my regular mirrored version of the live site.

Edited by szabesz
Link to comment
Share on other sites

Right now, I use rsync for code deployement and about to install a bash script running mysqldump and mysql for DB backup and deployment. I run Procache so in the midst of writing some php modules that reconfigure the site from Dev to Prod.

Hopefully I'll have a script that ties everything together soon.

If this is of interest to anyone, I could post it once I'm sufficiently happy with it?



  • Like 2
Link to comment
Share on other sites

19 hours ago, thlinna said:

..., but the use of it felt quite complicated, especially for newbie and for beta status. I hope no-one would not take this statement offending, but I feel sometimes tutorials and forum thread information require more indepth understanding of PW itself.

That's mostly because it's not a simple problem to solve at all. Merging of changes happening in two independent systems cannot work simply by clicking a button. There are various ways to make it work, which all come with their own kind of backdraws. With each project having different goals/needs it's a matter if choosing the one giving you the least pains, while helping you enough to make it worth. 

Also I'm not sure if I missed some beta statement in my modules topic, but it's really stable. Any upcoming changes should only affect the accompanied cli tool. Also I've it in production usage for over a year now with no issues in the latest months.

  • Like 5
Link to comment
Share on other sites

6 minutes ago, LostKobrakai said:

Also I'm not sure if I missed some beta statement in my modules topic, but it's really stable.

You might want to remove the beta status. No software is perfect, however if it is stable, why not give it 1.0.0? :) 

  • Like 2
Link to comment
Share on other sites

4 hours ago, LostKobrakai said:

Also I'm not sure if I missed some beta statement in my modules topic, but it's really stable. Any upcoming changes should only affect the accompanied cli tool. Also I've it in production usage for over a year now with no issues in the latest months.

Oops. My bad, I apologize. My assumption was based on some old forum threads. 

Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    • No registered users viewing this page.
  • Similar Content

    • By Jeannette
      Hello Fellows,

      I have a wonderful webshop, using PL and Padopt, a module developed by Julien Vaubourg and highlighted in ProcessWire Newsletter. Here is our shop: cheval-ami.fr
      I am looking for a new developer, helping me to keep the shop smooth and up to date.
      We would like to improve automated invoicing and worldwide shipping, getting DHL integrated and give people the possibility to create their personal accounts and be the nicest shop in the equine industry. We are still a young company, working all in german, english and french. We are very interested in a long time, kind and friendly relationship without any stress, but reliable reactivity in case of urgency. 
      We are based in Hamburg and near Paris and would prefere to work with somebody who is available to catch up around a table, at least sometimes.
      Contact: mail@cheval-ami.fr or +33 626 22 1000 or +49 170 20 28 215
      Thank you, Have a beautiful day,
    • By Kiwi Chris
      I don't really have anything public to show, as nearly all the work is back-end, but I thought I'd post here anyway as it's a pretty good example of just how powerful Processwire can be.
      About a year ago, I inherited an incomplete Craft website made by a designer. Apart from the usual company information site, it was supposed to provide a customer portal for clients of a wine bottling company to make bookings for production runs.
      Data for stock levels of goods like bottles and labels was to come from an inventory management system Unleashed https://www.unleashedsoftware.com/
      Unleashed provides a REST API, so I had to integrate with that first by writing an API integration module, and then ended up using Processwire's core lazy cron module to periodically pull data from Unleashed using a custom module.
      The booking forms have a lot of conditional fields, eg if you are bottling a given wine variety, you should only be able to select labels that match that variety. All this conditional stuff was achieved with a lot of additions to ready.php.
      I also needed to be able to created a predefined set of pages when a new user is added if they have a 'client' role. Once again, more hooking in ready.php
      I've used the Admin Restrict Branch module so clients can only see their own records when they're logged in, but staff can see all records.
      Lister Pro provides the ability to search and view completed production runs.
      Part way through the project, as the client was happy with the way things were going, I was asked to add in logistics and dispatch which is provided by another company, which also runs Unleashed with a separate set of data, and with some clients who don't bottle wine, but will end up using the same portal, so using the roles and permissions inherent in Processwire, I set up production templates with separate roles to dispatch templates, so I could easily have clients assigned access to just the templates they need.
      Tracey Debugger got a thorough workout along the way, and the debugger console is an absolute killer tool for making quick changes to data when updating a live site to match changes from the dev site.
      At the start of this project, I'd used Processwire quite a bit, but never dived into module development or hooking, but I've now ended up with a reasonable idea how they work.
      @bernhard has produced some excellent tutorials which I found really helpful figuring out how to create modules, and other people like @Robin S have answered questions when I've got stuck. @ryan himself has been helpful when I've been trying to do things that push either the limits of my knowledge or Processwire or both 😋  .
      Could I have done this with other tools? Depends. Wordpress would have been as useless as using petrol to fight a fire, however something like ASP.Net COULD have done the job but would have probably made things a lot more complicated. In parallel, I've been working on building a REST API with ASP.Net for another client to integrate with an existing SQL Server database, and I've found that Visual Studio is inclined to break projects quite regularly, with dependencies getting messed up, or even whole configuration files getting corrupted when it has a hissy fit, so working with Processwire is a pleasure in comparison.
    • By breezer
      Hello all 🙂
      First I would like to say PW is a wonderful CMS and a very helpful and friendly community, which is rare these days.
      I was doing web dev as a hobby while I cared for elderly parents but now it's time to move forward and get serious ( it's getting cold here so I'll be trapped indoors lol ). A while back I created a full featured built in forum system for Evolution CMS which I could rewrite for PW if there is enough interest, although it would have to be a paid module. At the moment I only have a working version on my local wamp, I can put it online next week and send anyone interested a link via PM.
      What do ya'll think?
      I removed the old screenshots as they are not relevant any more 🙂
    • By MoritzLost
      UPDATE: I have published a stable version of this module!
      Discussion thread:
      Github: https://github.com/MoritzLost/TextformatterPageTitleLinks
      Hello there,
      I'm working on a tiny textformatter module that searches the text for titles of other pages on your site and creates hyperlinks to them. I'm not sure if something like this exists already, but I haven't found anything in the module directory, so I wrote my own solution 🙂
      It's not properly tested yet and is still missing some functionality I would like to implement, so at the moment it should be considered in BETA. Features include limiting the pages that will get searched by template, and adding a custom CSS class to the generated hyperlinks. As I'm writing this I noticed that it will probably include unpublished and hidden pages at the moment, so yeah ... it's still in development alright 😅
      You can download the module from Github:
      There's some more information in the readme as well.
      Anyway, let me know what you think! I'm happy about any feedback, possible improvements or ideas on how to improve the module. Cheers.
    • By Guy Incognito
      Just wondering what software/approaches others take to the management of their code snippets.
      Relatively new to PW and finding that its logical approach is making it far easier for me to reuse code in projects and encourages me to try and be more organised!
      I've looked at few notes apps that piggyback of Gists - although I'm not sure if Gists is a good idea for private project work seeing as they're never totally private.
  • Create New...