Best way to move a Processwire site to a new server 2020?

[modifiedcontent]

What is currently the best way to move a Processwire site to a new server?

I have to upgrade my VPS, from CentOS 6 to CentOS 8. I have several relatively complicated Processwire websites. I am currently downloading all the files via FTP. I will export databases next.

I guess I'll use Export Site Profile. Or had that been replaced by something else? New modules that I missed?

Or will the FTP'ed files + exported database be enough? Just copy them 1-on-1 to the new server and it will all work?

What can go wrong? What are the pitfalls to watch out for?

 

[rick]

The procedure I use is:

Old Server:

1. Download the database export.
2. Download the site folder (you can skip cache).

New Server:

1. Install Processwire (same version).
2. Delete the site folder and Empty the database.
3. Upload the site folder from the old host.
4. Import the database from the old host.
5. Empty cache and sessions.
6. Update the config.php where necessary (database credentials, etc.).

Creating a site profile is not necessary.

[edit]
Since all my machines are *nix based, I use rsync to maintain duplicates:
 

rsync -azP host:source destination

 

[wbmnfktr]

I did just that with the DEV version of the Duplicator module. It went super smooth.

1. Install the module (dev version)
2. change some settings
3. create an export
4. download installer and archive
5. upload installer and archive
6. done

All you need is a new database and of course access to upload those files.

[modifiedcontent]

Excellent. Thanks rick and wbmnfktr. I'll do this in the coming weeks and will report back. 

[bernhard]

21 hours ago, modifiedcontent said:

Or will the FTP'ed files + exported database be enough? Just copy them 1-on-1 to the new server and it will all work?

That should be enough. You just need to make sure that the server setup works, so I'd install a fresh copy of PW manually to see if that works. Then you create the db user on the new server, restore the db, copy your files and you should be up and running.

I'm not saying anything against the other 2 recommendations. Just answering your question ? 

[modifiedcontent]

A few more points to watch out for. You need to copy the Authentication Salt in config.php to be able to use your login from the old database:

/**
 * Installer: User Authentication Salt 
 * 
 * Must be retained if you migrate your site from one server to another
 * 
 */

Installed modules and customizations can also mess up things. After simply uploading my backed-up /site folder into the new installation, I got a scary fatal server error. 

To step back I restored the /site folder of the fresh installation. I had replaced the fresh install database with my original backed-up database. That worked fine. I see all my old content and templates etc. in the admin area. 

Next I have uploaded all modules from the backed-up /site/modules to the new installation /site/modules. In the back-end I have clicked Refresh in Modules. It finds the modules and installs them. You may have to repeat Refresh a few times. 

I still keep getting this error:

SQLSTATE[42S22]: Column not found: 1054 Unknown column 'field_headshot.filedata' in 'field list'

After uploading the rest of the backed-up /site folder, everything else seems to work fine.

[Mantas]

On 12/8/2019 at 5:17 PM, rick said:

 

1. Empty cache and sessions.

What would be the best way to empty those?

[DV-JF]

Hey @Mantas, welcome to the forum and to ProcessWire ?

2 hours ago, Mantas said:

What would be the best way to empty those?

To do so I just delete the following folders via FTP:

/site/assets/cache/*/*.*
/site/assets/sessions/*/*.*

and truncate the table "caches" in the DB.

After that you have to relogin and do a "module refresh" via  "/processwire/module/?reset=1".

That's the way I do it.

[jon9china]

What if you are moving a site to a new server but changing domain name in the process?  Are there some other steps that are required?

[Gideon So]

Hi @jon9china

Nothing special to switch domain. Just remember to edit your config.php file and change $config->httpHosts to your new domain.

$config->httpHosts = array('yournewdomain.com');

Gideon

[BIMAQ-Admin]

On 12/8/2019 at 4:17 PM, rick said:

The procedure I use is:

Old Server:

1. Download the database export.
2. Download the site folder (you can skip cache).

New Server:

1. Install Processwire (same version).
2. Delete the site folder and Empty the database.
3. Upload the site folder from the old host.
4. Import the database from the old host.
5. Empty cache and sessions.
6. Update the config.php where necessary (database credentials, etc.).

Creating a site profile is not necessary.

[edit]
Since all my machines are *nix based, I use rsync to maintain duplicates:
 

rsync -azP host:source destination

 

I tried out this procedure with PW 3.0.229 at old and new server, but there are still 2 Problems:
1. header and footer are not output despite I modified the new site/config.php accordingly:

// Prepend this file in /site/templates/ to any rendered template files
// wb240816: $config->prependTemplateFile = '_init.php';
$config->prependTemplateFile = '_head.php';

// Append this file in /site/templates/ to any rendered template files
// wb240816: $config->appendTemplateFile = '_main.php';
$config->appendTemplateFile = '_foot.php';

and made shure, they are in the same subdirectory on both systems:

root@bimadmin-www:/var/www/html/processwire/site/templates/includes# ls -l
total 112
-rw-rw-r-- 1 www-data www-data  2445 Aug 16 12:10 _foot.php
-rw-rw-r-- 1 www-data www-data 16906 Aug 16 12:10 _head.php

2. Since the migration I cannot longer login as PW-admin, getting the error message: "admin - Login failed"
Directly before the migration I defined another PW-user, which also cannot login any longer!

The database seems to work, as the main content is displayed, you can try it out yourself with follwing links:
Old Server: https://www.bimaq.de/ , New Server: https://web.bimaq.uni-bremen.de/

Does anybody have an idea what the reasons coud be and how to fix the problems?

[wbmnfktr]

40 minutes ago, BIMAQ-Admin said:

// Prepend this file in /site/templates/ to any rendered template files
// wb240816: $config->prependTemplateFile = '_init.php';
$config->prependTemplateFile = '_head.php';

// Append this file in /site/templates/ to any rendered template files
// wb240816: $config->appendTemplateFile = '_main.php';
$config->appendTemplateFile = '_foot.php';

This doesn't look right. Maybe someone thought it would be a good idea but usually the other files (commented out) would be used here. However those _head and _foot are in /site/templates/includes/ therefore it's missing in your config.

// should probably be this:
$config->prependTemplateFile = 'includes/_head.php';
$config->appendTemplateFile = 'includes/_foot.php';

 

44 minutes ago, BIMAQ-Admin said:

New Server: https://web.bimaq.uni-bremen.de/

The new server throws errors that should be logged to /site/assets/logs/errors.txt.

Do you have still access to the old version?

[da²]

1 hour ago, BIMAQ-Admin said:

Since the migration I cannot longer login as PW-admin, getting the error message: "admin - Login failed"
Directly before the migration I defined another PW-user, which also cannot login any longer!

@BIMAQ-Admin Take care that this properties in the destination site/config.php are the same as the original site:

$config->userAuthSalt = 'xxx';
$config->tableSalt = 'xxx';

 

1 hour ago, BIMAQ-Admin said:

I tried out this procedure

The procedure could be more simple, no need to install PW then delete the database. Just rsync the files between 2 servers (with some exclusions for sessions and cache), and use mysql_dump to copy database from old server to new one.

This is how I replicate my dev environment to the staging server, except I also exclude site/config.php after it was initially uploaded and adapted to server config.

Example of database copy from source server to destination, to run on source server:

mysqldump --add-drop-database -uSQL_USER_SOURCE -pSQL_PASSWORD_SOURCE --databases DATABASE_NAME | ssh MY_SSH_USER@DESTINATION_SERVER_IP "mysql -uSQL_USER_DESTINATION -pSQL_PASSWORD_DESTINATION"

 

rsync, to run on source server (BUILD_DIRECTORY is the main PW directory path, /var/www/SITE_NAME/html/ is the destination directory:

rsync -avh --delete-delay --exclude-from=deploy-excludes.txt -e ssh BUILD_DIRECTORY SSH_USER@DESTINATION_SERVER_IP:/var/www/SITE_NAME/html/

 

deploy-excludes.txt:

site/assets/cache/*
site/assets/sessions/*

I'm not 100 % sure about syntax for "cache/*", I use "site/assets/cache/" in my case. Idea in your case is to copy empty folder without content, in case PW doesn't create it at runtime.

Note that I added a SSH key on destination server, it's why I don't need to specify SSH user and password in commands.

Edited August 16 by da²

[BIMAQ-Admin]

On 8/16/2024 at 5:50 PM, wbmnfktr said:

This doesn't look right. Maybe someone thought it would be a good idea but usually the other files (commented out) would be used here. However those _head and _foot are in /site/templates/includes/ therefore it's missing in your config.

// should probably be this:
$config->prependTemplateFile = 'includes/_head.php';
$config->appendTemplateFile = 'includes/_foot.php';

Thank you - of course this was a silly mistake, but not the main one.
The most important mistake was no connection to the database:
At the old system database name is 'processwire' and database user is 'processwire', at the new installation database name is 'processdb' and database user is 'processuser' by default.
Last week I made a database dump from the 'processwire' database and imported this dump into the new 'processdb' database (with new user).

Today I added a new database with the old name and user, imported the same dump and copied the old site/config.sys with the old database configuration but new server names into the new site folder:
Now it looks like the database connection is working, as I can login in the PW web interface, where I find the complete Page Tree, users and so on.
But there is still an Fatal Error maybe due to different php-versions and/or database-versions?:
Old server: PHP 7.4.33 (cli) (built: Jun  5 2024 05:05:14) ( NTS ) + mysql  Ver 15.1 Distrib 5.5.68-MariaDB, for Linux (x86_64) using readline 5.1
(PHP 7.4.33 is the newest version which can be installed in centos 7)
New server: PHP 8.1.2-1ubuntu2.18 (cli) (built: Jun 14 2024 15:52:55) (NTS) + mysql  Ver 15.1 Distrib 10.6.18-MariaDB, for debian-linux-gnu (x86_64) using  EditLine wrapper

I also tried out the new site/config.php with old database configuration and corrected prepend- and append-variables, but the result was worse.

Attached are screenshots of the homepage with errors and the line where the Fatal Error occured - I have no idea how to fix it.

[Jan Romero]

Before PHP 8 you could put anything into the count() function and it would return 1 if it wasn’t null. The code in line 241 in includes/_head.php depends on this old behaviour, so you’ll have to rewrite it.

Since there only ever seems to be a single image or none at all, you can probably just remove count() entirely:

if ($page->eyecatcher) { … }

There may be more instances of similar things in the codebase as PHP 8 deprecated several things. Shouldn’t be too hard to fix, though.

Edited August 22 by Jan Romero

[BIMAQ-Admin]

Indeed, after removing "count()" in this place the main page is displayed completely.

In the other pages there are still bugs, I will look for next week - Thank you for your help!

[BIMAQ-Admin]

Climbing through the whole site map, I found more instances of the "count("-type errors which could be successfully deleted, but there are two other errors: (maybe this is more a PHP than PW problem, but perhaps someone here has an idea of the reasons)

1. "Unknown Selector operator: '[empty]' -- was your selector value properly escaped? (in wire/core/Selectors.php line 222)",which seems to be caused by site/templates/publications.php, because it only occurs where this template is used (-> PW-publications-error.PNG).
The corresponding lines of "publications.php" are:

Quote

36           $sortByYear = "sort=-title";
37            $allPubYears = $page->children($yearSelect == 0 ? $sortByYear : "title$yearSelect" . ", " . $sortByYear);
38            $allPubs = $allPubYears->children($selector == "" ? $default : "$selector");
39            $parent = $pages->get(1591); //Auswahlfelder Genres
40            $children = $parent->children;
41            $genre = [];
42            foreach($children as $child) {
43                array_push($genre, $child);
 44           }

 

2. " Uncaught TypeError: renderContactSmall(): Argument #1 ($contact) must be of type Page, bool given, called in site/templates/structure.php on line 25 and defined in site/templates/includes/functions.php:670" (-> PW-lfstroemung-err)

Quote

structure.php line 25ff:

                    echo renderContactSmall($contact);
                    if ($page->pdf_file) {
                        $pdf = $page->pdf_file;
                        echo "<a href='$pdf->url' target='_blank'>&raquo;&nbsp;&nbsp;" . __("Beschreibung als PDF") . "</a>";
                    }

functions.php line 670ff:

function renderContactSmall(Page $contact) {
    $out = "";
    $out .= "<div class='visible-xs''>" .
            "<h4>" . __("Kontakt") . "</h4>" .
            "<p><b><a href='$contact->url'>$contact->employee_first_name $contact->employee_last_name</b></a></br>" .
            "E-Mail: $contact->employee_mail</br>" .
            __("Telefon: ") . "<small>+49 (0)421 218 </small><b>646 $contact->employee_phone</b></p>" .
            "</div>";
    return $out;
}

The URL of the still active old webserver is: https://www.bimaq.de,
the URL of the new webserver is: https://web.bimaq.uni-bremen.de
 

[Jan Romero]

This is difficult to fix without more of the surrounding code. This isn’t on Github by any chance? If you’re not comfortable with fixing it yourself, you should probably give someone with expertise in PHP or ProcessWire direct access.

Also please use the code formatting to post snippets, instead of quotes.

Try to see if you can find where $contact is declared in the first 25 lines of structure.php. Apparently it’s a Boolean, but is expected to be a Page.

[da²]

2 hours ago, BIMAQ-Admin said:

Uncaught TypeError: renderContactSmall(): Argument #1 ($contact) must be of type Page, bool given

I bet $contact is coming from a page reference field that is configured to return false when empty. So if $contact is optional, the code should verify its value before to use it and call renderContactSmall().

For the empty selector, it's necessary to investigate the code that generates it, and log value of variables it's using.

[BIMAQ-Admin]

On 8/29/2024 at 5:19 PM, da² said:

For the empty selector, it's necessary to investigate the code that generates it, and log value of variables it's using.

How can I log value of variables used in a template?

I now attach the complete template "publications.php":
On the old server (which uses PHP 7.4.33) it works fine, the new server (PHP 8.1.2-1ubuntu2.18) throws the error message, which is defined in wire/core/Selectors.php:
 

        public function create($field, $operator, $value) {
                $not = false;
                if(!isset(self::$selectorTypes[$operator])) {
                        // unrecognized operator, see if it's an alternate placement for NOT "!" statement
                        $op = ltrim("$operator", '!');
                        if(isset(self::$selectorTypes[$op])) {
                                $operator = $op;
                                $not = true;
                        } else {
                                if(is_array($value)) $value = implode('|', $value);
                                if(is_array($field)) $field = implode('|', $field);
                                $debug = $this->wire()->config->debug ? "field='$field', value='$value', selector: '$this->selectorStr'" : "";
                                if(empty($operator)) $operator = '[empty]';
                                throw new WireException("Unknown Selector operator: '$operator' -- was your selector value properly escaped? $debug");
                        }
                }
                $class = wireClassName(self::$selectorTypes[$operator], true);
                /** @var Selector $selector */
                $selector = $this->wire(new $class($field, $value));
                if($not) $selector->not = true;
                return $selector;

As I upgraded PW before the migration also wire/core/Selectors.php is exactly the same on both servers.

[da²]

You can log values with wire()->log->message("foo"), or with a var_dump() followed by an exit().

But here the error message is saying everything:

On 8/29/2024 at 3:02 PM, BIMAQ-Admin said:

"Unknown Selector operator: '[empty]' -- was your selector value properly escaped?

  And the code:

1 hour ago, BIMAQ-Admin said:

if(empty($operator)) $operator = '[empty]';
throw new WireException("Unknown Selector operator: '$operator' -- was your selector...

So maybe the problem comes before this function call, check the parameters values, especially $operator.

Edited September 5 by da²

[BIMAQ-Admin]

19 hours ago, da² said:

You can log values with wire()->log->message("foo"), or with a var_dump() followed by an exit().

Sorry for the (maybe silly) question, I am not so familiar with php:
Are this lines to add in the template I want to debug?

What is still puzzling me, is that the error only occurs with php V8.1.2 and not with php V7.4.33

[da²]

public function create($field, $operator, $value) {
  var_dump($operator);
  exit();

You could also use a PHP debugger but it needs some configuration.

14 minutes ago, BIMAQ-Admin said:

What is still puzzling me, is that the error only occurs with php V8.1.2 and not with php V7.4.33

Yes there's probably some code doing a deprecated thing (in PHP 8.1), and not getting the expected result, maybe a code that prepare the $operator string. You may also check if you have some warnings or notices about deprecated usages, by using this handler in site/init.php. Then you'll find the logs in PW > logs > php-warnings.

set_error_handler(
    function (int $errorNumber, string $message, string $file, int $lineNumber): bool {
        $errorString = "File \"$file\", line $lineNumber => $message";
        
        wire()->log->save('php-warnings', $errorString);

        return true;
    },
    E_WARNING | E_NOTICE);

 

Edited September 6 by da²

[BIMAQ-Admin]

Thank you da², that was very helpful!

I am not yet finished, but I could locate the problems quite closely using your tips.

[JerryDi]

I've followed the process meticulously to manually move to a new server
I now find that my login won't work despite replacing the config file from the original install

Any advice please?

thanks

J