Moving PW between staging and live server

[Adam Kiss]

Hi Ryan,

could you write a little tutorial about how to safely move PW2 Install from staging server to live server?

Thank you,

Adam

[ryan]

could you write a little tutorial about how to safely move PW2 Install from staging server to live server?

That's a good idea, though I'm not sure that I've found an ideal method. I should probably be using Git for this, but I don't yet know it well enough. When I referred to deployment in the other messages, I was talking about deploying the source code to Git, rather than deploying a staging to production site.

But here's how I do it. For live sites, I use rsync to deploy my staging files to the server's files. i.e. at the OS X command line:

rsync --archive --rsh=/usr/bin/ssh --verbose site/templates/* user@somehost.com:www/site/templates/

This is made easier if your SSH keys are setup so that you don't have to type your password every time you do this.

I selectively rsync the /site/templates/ or /wire/ dirs, depending on what I'm upgrading. If I'm upgrading the version of ProcessWire, then I rsync the /wire/ dir. If I'm upgrading the site's functionality (usually via it's templates, stylesheets and scripts) then I'll rsync /site/templates/.

I don't do full-site rsyncs because the version of data on my dev server is usually older than the version of data on the live site. Though I do daily backups, I don't mirror my dev copy's data with the server's... unless there's a reason to do so. My deployments are usually code related, not database related. So to safely deploy upgrades to a live site, I specifically avoid rsyncing any files in /site/assets/ or any database files. Instead, I just keep rotating backups of those and manually upgrade my dev site to the latest version of that data whenever it makes sense to do so.

Even though I'm using rsync, which makes this all a lot nicer, quicker and more secure, there's not a major difference between this method and just FTPing the files. So you can see I don't have a perfect system for this. I'm interested to learn other people's methods.

[Adam Kiss]

I was looking for the first time move also – what do we do?

copy everything on server, create backup of staging database and then? what files to change (because database settings of server, where PW2 was installed and developed (either dev or local) probably has different settings than live server)

And in addition, there are people (like me), who use shared hosting for some sites (in here, it's much more efficient in speed, price and reliable than cheap virtual), so they might not be ssh

but the part of copying data between servers is the less important – moving from staging to live for the first time it's important! (i.e. in wordpress, you have to backup, edit and copy files in selected order, otherwise you have a problem)

Adam

[ryan]

MIGRATING A SITE FROM DEV TO LIVE FOR THE FIRST TIME

STEP 1: Copy all the files from dev to live

---

Copy all the files from your dev installation to the live installation. Whether you use FTP, rsync, scp or some other tool doesn't matter. What does matter is that you copy everything and that you retain the file permissions–More specifically, make sure that /site/assets/ and everything in it is writable to the web server. Here are some sample commands that would do it (replace 'local-www' and 'remote-www' with your own directories):

Using rsync:

rsync --archive --rsh=/usr/bin/ssh --verbose /local-www/* user@somehost.com:remote-www/

Using scp:

scp -r -p /local-www/* user@somehost.com:remote-www/

Using FTP/SFTP/FTPS:

I think this will depend on the FTP client that you are using as to whether it will retain the permissions of the files you transfer. Hopefully it will do that by default, if not, check if there is a setting you can enable. If not, then you may have to adjust the permissions manually in /site/assets/ and the dirs/files in there.

Make sure /.htaccess got copied over too

Depending on how you copied the files in step 1, it may or may not have included the /.htaccess file. Make sure that gets copied over, as many copying tools will ignore hidden files by default.

STEP 2: Transfer the MySQL Database from Dev to Live

---

a. Export dev database

Export ProcessWire's database on your development server to a MySQL dump file. PhpMyAdmin or mysqldump are the most common ways to do it. Personally I use PhpMyAdmin because it's so simple, but here is how you would do it with mysqldump:

mysqldump -u[db_user] -p[db_pass] [db_name] > site.sql

b. Create live database

Create a new MySQL database and database user on the live server and make note of the DB name, DB host, DB user and DB pass, as you'll need them.

c. Import dev database to live

Import the MySQL dump file you exported from your dev server. Most web hosts have PhpMyAdmin, so that's what I use to import. If you have SSH access, you can also import with the mysql command line client, i.e.

mysql -u[db_user] -p[db_pass] -h[db_host] [db_name] < site.sql 

d. Update /site/config.php

On the live server, edit the /site/config.php file. At the bottom you will see the database settings. Update these settings to be consistent with the database/user you created, then save. Most likely you will only be updating these settings:

$config->db_name = "database name"; 
$config->db_user = "database user name"; 
$config->db_pass = "database user password";
$config->db_host = "database host, most commonly localhost";

STEP 3: Test the Site

---

Your site should now be functional when you load it in your browser. If not, then enable debug mode:

a. Turn on debug mode (only if you get an error)

Edit /site/config.php and look for $config->debug = false, and change it to $config->debug = true. Then load the site in your browser again and it should give you more details about what error occurred. If you aren't able to resolve it, contact Ryan.

b. Browse the site

Assuming your site is now functional, browse the site and the admin and make sure that everything looks as it should. If it looks like stylesheets aren't loading or images are missing, it may be displaying cached versions from your dev site. If that's the case, you need to clear your cache:

c. Clear your cache (optional)

Login to the ProcessWire admin and go to Modules > Page Render > and then check the box to "clear page render disk cache", then click save. This should resolve any display issues with the site's pages.

d. Practice good housekeeping

Make sure that you've removed /install.php and /site/install/, and check to make sure that /site/config.php is not writable. While this isn't absolutely necessary, it's good insurance.

[ryan]

Adam, writing this up got me to thinking that I may be able to write a tool that would automate most of deployment, by creating an installation profile. The tool would be in the ProcessWire admin, and when you run it, it would create a installation profile of your site. Then, you would just upload the profile to your server, and then load it in your browser (which would initiate the installer). I kind of like this idea because it would handle any file permissions and export + import of the database automatically. Are you aware of any other CMSs that do something like this? (I'm not). I'll put more thought into this.

[apeisa]

Adam, writing this up got me to thinking that I may be able to write a tool that would automate most of deployment, by creating an installation profile. The tool would be in the ProcessWire admin, and when you run it, it would create a installation profile of your site. Then, you would just upload the profile to your server, and then load it in your browser (which would initiate the installer). I kind of like this idea because it would handle any file permissions and export + import of the database automatically. Are you aware of any other CMSs that do something like this? (I'm not). I'll put more thought into this.

That would be perfect! I think installation profiles would be very cool thing for showcasing also. It would be nice to show ready & easy to install packages for ie. portfolio sites, product catalogs, blogs etc...

I don't know how easy it is to export "ensemble", but Symphony has something similar: http://symphony-cms.com/download/ensembles/

[neotoxic]

Just a thought but given similarity of the task, the ability to export and install a site profile could be taken a step further to facilitate a nice clean and comprehensive back-up process. Allowing both a sites structure and or its data to be archived. Backed up and recovered as needed.

[ryan]

I agree. I think this is something we'll definitely have to do. Would be great to have a 1-step backup that includes database and site files. Though it could be a slow process for very large sites.

[apeisa]

I got strange problem when migrating one of my personal projects from local (my win7 laptop) to server (linode vps). I cannot login. Strange thing is that if I put correct username and password, it doesn't say nothing, just reloads the login page. If I have wrong password, then it says "Login failed for apeisa".

I have checked that my db:s are identical, also config files are identical (only db settings differ). $config->userAuthSalt is identical, as are field_pass hash and salt.

I even tried to change my superuser password through api. It changes it, but same thing at login, silent death

(no hurry with this one, just wondering what might be an issue here)

[Soma]

I encountered a similar problem when putting a local codeigniter project to a server. I think I had to generate a new user pwd, although using same name/pw... and it worked again. but never couldn't make up what the problem was,must have been a encryption key / salt problem somehow...

[ryan]

Antti, what version of PHP?

[apeisa]

On my local 5.3.3 and on server it is 5.3.2-1ubuntu4.9

[ryan]

I was originally thinking maybe you had a version prior to 5.2.4 because I had the same problem on a PHP version older than that (one of the reasons why PW doesn't support versions before 5.2.4). It doesn't sound like that's the case here... A couple more things to try:

1. Look in /site/assets/logs/errors.txt to see if there is anything in there.

2. Try installing a fresh copy of PW under some subdirectory, just to see what the installer says. The installer might find an issue that wouldn't be apparent when migrating an existing site. 

[apeisa]

1. Only missing favicon.ico

2. Installed just fine and login was successful. Strange. Something must be wrong with my db import/export. I will try that again.

[ryan]

Double check that you've got the .htaccess file in your webroot. The favicon.ico messages shouldn't appear in your error log because the .htaccess file prevents them from going to PW. So if there is a favicon.ico message there, it makes me think the .htaccess might be missing or is from an older version of PW?

[apeisa]

I think we are getting closer. I did run older version of pw on the same dir some weeks ago. I removed that with winscp and that have most probably left some hidden files like .htaccess. Stupid me didn't check if there was older .htaccess (I just overwrote it with the correct). No change here.

Now I jumped to shell and removed with rm -rf the whole dir and started re-transferring now.

[apeisa]

Now it works. Not sure about the exact reason, but I think I add notice to myself: make sure the folder where you are transfer your files really is clean...

[ryan]

Glad to hear it's working after a re-upload. I think it was most likely just that htaccess file… That's the only unix-hidden file in PW.

[apeisa]

Yep, that it was. Now that I rethink it, I actually copied the new .htaccess from the new site I created on subdir to test if it works - and not from the local site that where I should have copied it. And funny thing is that the test site in subdir was most probably build from same version of P21 than the "ghost site" that I removed in first place. So when I overwrote the .htaccess, it was exactly same contents...

Thanks for your help Ryan!

[doolak]

Hi there,

i have the same problem when tried to move a site from dev to live server.

I have reloaded the .htaccess, upgraded to the last version but the login doesnt work.

I installed PW in another subdirectory and that fresh install works fine. As i wanted to test if it has sometghing to do with the encryption i copied field_pass from the database of the new installation to the live site and the userAuthSalt from the config - but still the same problem: the login page just reloads.

Any other idea?

Kind regards,

Christian

=====

Edit:

I found the problem - the permissions on the folder site/assets/ and on the site/config where not correct - i changed them and the login workes fine now.

[Natetronn]

Does the PW "installation profile" idea now exist?

Is this it here? - http://modules.processwire.com/modules/process-export-profile/

Thanks

[Soma]

I'm not sure what you mean by "does it now exist?".

This Profile Exporter module exists since a long time. People are using to export a basic setup, this start with that on a new project. Some people are building a shop site profile and so on. Ryan has for example done a great Blog profile ready to install and use.

[ryan]

The profile exporter does exist, but I don't think it has seen a lot of action yet. I hope to change that and put out a couple more profiles myself here soon.

[devcow]

I can reproduce the strange condition with the password salt en- and decryption.

I depends on the php version how they use the keys for decryption.

i had on my dev machine another php version and wanted to migrate the build site to my customer and couldn't log in.

My work around was -> Lost Password Function

I could reset the password and now i am in.

I will not think off the possibility the mail relay would not be working ... :-(

It took me today some time to figure it out but now it works fine.

i used the version on dev machine -> PHP 5.3.10-1ubuntu3.8 with Suhosin-Patch (cli)

Live machine -> PHP 5.2.17

I hope this helps somebody! ;-)

Greetz,

Fabian

[ryan]

i used the version on dev machine -> PHP 5.3.10-1ubuntu3.8 with Suhosin-Patch (cli)

Live machine -> PHP 5.2.17

Unfortunately once you start on a PHP 5.3+ you can't migrate passwords back to a PHP 5.2 server. PHP 5.2 did not support blowfish, so it has no way to validate the password. The only solution is to do as you did, and reset your password.