☁️ Duplicator: Backup and move sites

[flydev]

Hi @PavelRadvan thanks.

What you read in VSCode is just a warning, so no problem.

I just opened an issue on github to remind me of it, I will write the php 8.2 upgrade at the same time of wire-cli.

About the PW Directory, no idea, I will look at it, but the last stable version is v1.5.2 ?

 

[Pavel Radvan]

Hi.
Thanks for answer. I have another project which is based on PHP 8.3 and I think there is no warning in vscode...
Regarding version and updates I am little bit confused with this:
Screenshot from PW module repository:

Screenshot from github:

Which is right ? github or pw in date of updated ? Is module on pw repository same with content or on github...?

[flydev]

@PavelRadvan the module - from the PW Directory - is pulled from Github, so it's the same content / version. The update field you see on the PW Directory, is the last time of the pull.

[Pavel Radvan]

OK. Now I understand...so that is only github update check date...I see it is on other modules too...

[heybran]

Hi,

New to ProcessWire and definitely new to Duplicator, I'm helping a non-profit organization to backup their site, but when I click submit inside Duplicator module, no error is thrown so I think it looks good, but also no success indicator that the backup is started or in process.

I'm on Duplicator version 1.5.2

Wondering if anyone has previously recorded an video tutorial to backup ProcessWire site with Duplicator?

 

Thanks,

Brandon

[heybran]

I've SSH into server and run: `php applications/{appname}/public_html/site/modules/Duplicator/cron.php` and it seems to work. 

 

[netcarver]

@heybran Hello Brandon, welcome to the forums. There's a video in the opening post of this topic that shows you the basics. There's another one here. There's no need to SSH in and manually kick the cron script.  Hope those get you started.

Also, please could you edit your post above and remove the full-quote of your previous post - it's not needed and is something we want to discourage in these forums.  Cherry picking relevant sections of a previous post for quotation is fine - but full-quoting whole previous posts makes your account look like a spammers.

[heybran]

Hello @netcarver, thanks for the heads-up, I will keep in mind. Although I don't see any button that allows me to edit my initial post, that's why I follow up with that comment yesterday. Is there something I missed about how to edit post? Thanks

[netcarver]

@heybranThanks for letting me know, Brandon, I was unaware of that limitation - it's probably as you are a new user on the forum. I've edited the post to remove the quote.

Did you manage to get things working via the web interface?  Things have not changed much since the videos I linked were originally posted.

[heybran]

@netcarver Thanks for the update. I've deleted my staging site already, but I think it will work (I missed the `package` button before on the web interface, that's why I was confused). Will try it out on the organization's server next time if I get an chance to take a look at their production site.

[flydev]

@heybran you will find the direct access to Duplicator's packages management from menu Setup > Duplicator 

  

On 5/18/2024 at 6:56 AM, heybran said:

I've SSH into server

On server, you can list packages under  ls -lah /public_html/site/assets/backups

And welcome to the forum ?

[flydev]

@adrian I applied fixes you reported earlier. I also merged changes from the dev (stable as today - v1.5.4) into the feature-encryption branch, this last is not ready as I need to re-read the whole conversation I had with @netcarver in order to finish it, but it works, just in case you want to test it.

FYI, the v2 is also in the pipe and coming with a dedicated software to manage the process between all your websites from a centralized desktop/mobile app. It might will be part of a paid feature, still not decided.

[protro]

@flydev I am receiving a

General error: 1273 Unknown collation: 'utf8mb4_0900_ai_ci'

error when trying to migrate a PW site from a production/staging server back to local dev with MAMP on macOS. Is this a MySQL version error? And if so how do I fix ?

also getting this message which I have never seen in Duplicator just before inputting database name user/pass and extracting

Deprecated: strrpos(): Passing null to parameter #1 ($haystack) of type string is deprecated in /Applications/MAMP/htdocs/production-pulldown/installer.php on line 709

- - - SOLUTION - - -

I found a solution to my problem. I had to edit the .sql file within the duplicator-temp directory and do a search/replace all instances of utf8mb4_0900_ai_ci with utf8mb4_unicode_ci

That seemed to have solved the issue. Never had that happen before. Must be an incompatibility with the duplicator package generated on the staging server and my local dev setup.

Edited August 14 by protro

[wbmnfktr]

5 hours ago, protro said:

search/replace all instances of utf8mb4_0900_ai_ci with utf8mb4_unicode_ci

I have had this in the past or at least very similar with a custom Debian-based Apache, MySQL setup I tried to run and manage myself. I don't know what I did wrong, but everything seemed a bit off. Since using DDEV, moved away from Laragon (Win) and MAMP (Mac), never had this issue anymore.

The question to ask here is: Which system is off or needs a better config?

Where is your live server from or who is your hosting company?
If I had to guess... A2*, Dream*, or Blue*host - or something similar.

[flydev]

Hello guys. 

If you get a “Collation unknown” error, it's probably because your MySQL server version is earlier than 8.0.1, i.e. your dev/staging/prod servers are not the same (I suggest you harmonize and work on the same servers to avoid potential problems).

You can consult the mysql documentation on collation migration: collations-migrating-from-older-collations

If you'd like to go further in understanding the collation naming suffix, take a look at the table in the mysql documentation there.

(I take a note about the installer file; You can give a read to my comment here too for notice type meanings).

[MarkE]

Hi @flydev. Just running this module for the first time, so excuse any stupid questions. I created the zip (big - over 500mb, but the database itself is only 58mb) and put it and the installer in my new web root I then ran the installer - all the checks were OK but I got an error:

[08-Sep-2024 19:14:26 Europe/London] PHP Fatal error:  Uncaught ValueError: Invalid or uninitialized Zip object in /home/............./installer.php:736
Stack trace:
#0 /home/............../installer.php(736): ZipArchive->extractTo()
#1 /home/............../installer.php(105): Installer->extractPackage()
#2 /home/............../installer.php(2255): Installer->execute()
#3 {main}
  thrown in /home/.........../installer.php on line 736

I then tried to use the 'native' method, but got exactly the same error.

Any ideas?

[flydev]

Hi @MarkE there is no stupid question as they help others dev to get the right answer when they search for it.

What’s the type of hosting? Webhosting or you have your hands on the server? 
the error seem to be due to the size of the zip as it say the object is not initialized, but I might be wrong, and about the native method, maybe its an issue with the zip extension.

Please check the logs of the server and duplicator.

edit: I suggest you to try with small setup, I mean you exclude some folders from the backup to get a smaller package just for testing.

[da²]

@flydev I've played with ZipArchive recently and had some issues when unzipping on Linux (but not on Windows with 7zip), because I forgot to exit() PHP script after outputting headers and zip file. So I know a zip file can be opened on some software and not on other if it contains a problem.

For the @MarkE issue, I'm wondering if it could come from the 2 lines in Duplicator.module where you update the zip archive without providing a flag in $zip->open()?

Line 566 : if ($zip->open($pwbackup['zipfile']) !== true) {
Line 939 : if ($zip->open($zipfile) !== true) {

I don't really believe this solution, but I've seen a lot of posts like that when searching for my own issue: https://stackoverflow.com/a/17975009

Edited September 8 by da²

[MarkE]

2 hours ago, flydev said:

What’s the type of hosting? Webhosting or you have your hands on the server? 

I am moving the site from one web hosting service to another. I have full access via cPanel.

2 hours ago, flydev said:

Please check the logs of the server and duplicator.

The duplicator log indicates that everything was normal:

2024-09-08 18:18:31:  - job finished in 372.169112sec
2024-09-08 18:18:31:  - package saved in local folder: 2024-09-08_18-12-25-{........}.co.uk.package2.zip
2024-09-08 18:18:31:  - package built successfully in 372.167747sec
2024-09-08 18:12:19:  - Backup using standard mode
2024-09-08 18:12:19:  Backup Database

The built package was not openable in windows (including by 7zip) so it seems the fault is in the package not the installation.

2 hours ago, flydev said:

I suggest you to try with small setup, I mean you exclude some folders from the backup to get a smaller package just for testing.

I tried that - just including wire. No package was built.

1 hour ago, da² said:

I'm wondering if it could come from the 2 lines in Duplicator.module where you update the zip archive without providing a flag in $zip->open()?

Line 566 : if ($zip->open($pwbackup['zipfile']) !== true) {
Line 939 : if ($zip->open($zipfile) !== true) {

The fix could be to use:

$zip->open($file, ZIPARCHIVE::CREATE);

That didn't fix my problem. Interestingly, the package was twice as big.

I think that the problem may be due to the source installation being multi-site. There are two sites - site-ncog and site-music. I only want to duplicate and move site-ncog. There is no /site/ folder. The duplicator doesn't seem to cater explicitly for this (I thought it might duplicate both sites and then I would need to remove one). I see that Duplicator.module has 

const DUP_SITE_FOLDER = 'site';

I tried changing that to site-ncog, but no package was built.

I have a dev site that just has the one site (in a folder called /site/). I haven't tested it fully, but the package was built and was readable in Windows. So one option might be to fully replicate the live site on my dev environment and then duplicate that. This would have some complications with file names, which might be resolvable by renaming /site/ to /site-ncog/ after the duplication.

[flydev]

8 hours ago, MarkE said:

I think that the problem may be due to the source installation being multi-site.

Thats make sense then. I think we never had this case and as I never used this setup, we can assume it's not supported (you spotted the constant).  I am interested to get the log from the server that you can get on the CPanel.  

I open an issue on github, I will test it.

edit: @MarkE what is the version of PHP ?

edit2: I tested backup and restoration on a package of 600MB on php-8.2, no issue so far.  

8 hours ago, MarkE said:

I have a dev site that just has the one site (in a folder called /site/). I haven't tested it fully, but the package was built and was readable in Windows. So one option might be to fully replicate the live site on my dev environment and then duplicate that. This would have some complications with file names, which might be resolvable by renaming /site/ to /site-ncog/ after the duplication.

Yes ok, renaming to `site` should work if no hosting memory error is raised, which is common on those webhosting.

[MarkE]

2 hours ago, flydev said:

I am interested to get the log from the server that you can get on the CPanel.  

I'm afraid I am on a shared server so I only get error logs.

2 hours ago, flydev said:

what is the version of PHP ?

8.1

2 hours ago, flydev said:

Yes ok, renaming to `site` should work if no hosting memory error is raised, which is common on those webhosting.

Actually, I realise it is not as simple as that because all the database settings will be wrong. If I can't use duplicator, I may as well set up a new PW site, ftp the code and restore the database manually. It would be nice to get Duplicator working, though because it is really neat 🙂.

[flydev]

About the logs they are generally located in /public_html/error_log or /logs/example_domain_com.php.error.log

32 minutes ago, MarkE said:

because all the database settings will be wrong

The database is not altered, the instaler basically just import your database backup file as is.

 

40 minutes ago, MarkE said:

It would be nice to get Duplicator working, though because it is really neat

Yeah, logs would help, but I will try to setup a muti-site instance to see how it work. I know the installer is also looking for the hardcoded `site` location.

[MarkE]

1 hour ago, flydev said:

About the logs they are generally located in /public_html/error_log or /logs/example_domain_com.php.error.log

The only log I have with entries at the relevant time is the error_log created by duplicator which I quoted in my OP at 

Edit - that is from the target host. There was no relevant log on the source host.