☁️ Duplicator: Backup and move sites

[adrian]

Thanks @flydev - it's awesome to have that option - not sure about using a "every 1 minute (*/1)" call as the example though  

[adrian]

@flydev - I set up the system cron last night and didn't think too much about, but checking today I see that it didn't work. It turns out the issue is:

$e = new HookEvent();

You just need to add the ProcessWire namespace to the top of the cron.php file and everything works as expected.

[adrian]

Hi @flydev and @kixe - I am using Duplicator and CronjobDatabaseBackup on a site and I would like Duplicator to also send the DB backups created by CronjobDatabaseBackup to Google Drive. The problem is that because by default they both store files in site/assets/backups

My solution has been to move Duplicators backups to /site/assets/duplicator and then uncheck the "Backups" options under the "Files and folders excluded from the package" setting. This should work, but I think perhaps there could be a cleaner solution. Also @flydev - I think the warning about exponential size if unchecking "backups" should have the caveat that it is only relevant if backups are being stored under /sites/assets/backups.

The other quick question - if I set the max number of packages to 10, but leave the "Remove backup packages older than" option to "never", what will happen. I assume that as the 11th is added, the 1st will be removed, but this seems to contradict the "Remove backup packages older than" option. Can you explain exactly what happens here?

Edited December 6, 2017 by adrian
I'm an idiot :)

[adrian]

Sorry, I forgot that Duplicator of course also handles DB backup

[flydev]

19 hours ago, adrian said:

I think the warning about exponential size if unchecking "backups" should have the caveat that it is only relevant if backups are being stored under /sites/assets/backups

I will add this check, it make sense to me. Also it might be worth to check if the backup destination is being included - e.g. the destination is in the webroot but not excluded - then if yes, the warning appear, as the package will be saved and the ZIP file will grow.

 

19 hours ago, adrian said:

if I set the max number of packages to 10, but leave the "Remove backup packages older than" option to "never", what will happen.

As you said, the last package will be saved and the oldest package is removed if the limit is reach.

Now I am thinking to make those two settings checking each other giving the priority to the "Remove  backup packages older than" setting.

 

20 hours ago, adrian said:

You just need to add the ProcessWire namespace to the top of the cron.php

I will test it more closely as I would like to keep the compatibility to ProcessWire 2.x in case some people want backup their pw2 website for an eventual upgrade.

 

My brain is overheating today, I'll read again your comments into an hour or so

[adrian]

1 minute ago, flydev said:

I will test it more closely as I would like to keep the compatibility to ProcessWire 2.x in case some people want backup their pw2 website for an eventual upgrade.

Good point - then simply do this:

$e = class_exists("\ProcessWire\HookEvent") ? new \ProcessWire\HookEvent() : new HookEvent();

 

1 minute ago, flydev said:

My brain is overheating today, I'll read again your comments into an hour or so

I know that feeling very well - rest up!

[adrian]

Thinking about this a little more - is there any reason to actually instantiate a new HookEvent() ? The cronJob() method doesn't actually make use of the passed $event. Am I missing something?

[flydev]

I used it in an earlier version and I left it in case. I did well not to push the new version.

[theo]

Hi

I have a PW 2.7.2. installation.

According to to docs, your module should work with 2.7 resp. 2.7.2 but it does not install because of

const DUP_MIN_VERSION               = '2.7.3';

Is there a technical reason for this, or can I safely change the DUP_MIN_VERSION to '2.7.2' in the module source?

Thank you.

[flydev]

Hi @theo,

you can change it safely, the reason if I remember correctly, is because ProcessWire 2.7.3 was the latest stable version of the 2.x branch - I didn't tested Duplicator on a earlier PW version.

 

Please report here if you could backup it successfully

Edited December 7, 2017 by flydev
precision

[theo]

Thank you.

No, it does not work here right now, but I think it does not have to do with the PW version, because it does not work with 3.0.83 either.

Could it be because I'm using mod_userdir? https://httpd.apache.org/docs/2.4/mod/mod_userdir.html

Where in my case

File root of PW: /home/theo/public_html/pwdev/

Web root of PW: http://localhost/~theo/pwdev/

Thank you.

[horst]

Hi @theo , curious to know what you get when outputing something like

echo "<pre>
path  {$config->paths->templates}
url   {$config->urls->templates}
";

 

[flydev]

Also, could you check the logs please in /your_root_dir/site/assets/logs/duplicator.log to see if there is something revelant ?

[theo]

5 minutes ago, horst said:

Hi @theo , curious to know what you get when outputing something like

echo "<pre>
path  {$config->paths->templates}
url   {$config->urls->templates}
";

 

This:

path  /home/theo/public_html/pwdev/site/templates/
url   /~theo/pwdev/site/templates/

Thank you.

[theo]

6 minutes ago, flydev said:

Also, could you check the logs please in /your_root_dir/site/assets/logs/duplicator.log to see if there is something revelant ?

There is no such file. But there is a warning on the package manager page:

Warning: filesize(): stat failed for /home/theo/public_html/pwdev/site/assets/logs/duplicator.txt in /home/theo/public_html/pwdev/wire/core/FileLog.php on line 225

Warning: filesize(): stat failed for /home/theo/public_html/pwdev/site/assets/logs/duplicator.txt in /home/theo/public_html/pwdev/wire/core/FileLog.php on line 225

Warning: filesize(): stat failed for /home/theo/public_html/pwdev/site/assets/logs/duplicator.txt in /home/theo/public_html/pwdev/wire/core/FileLog.php on line 237

Warning: file(/home/theo/public_html/pwdev/site/assets/logs/duplicator.txt): failed to open stream: Datei oder Verzeichnis nicht gefunden in /home/theo/public_html/pwdev/wire/core/FileLog.php on line 238

And there is a mesage after pressing "Initiate Backup Process"

No route, document, custom route or redirect is matching the request: /~theo/pwdev/~theopwdevprocesswire/setup/duplicator/?action=none | Specific ERROR: No route matched the request

 

 

[flydev]

Thanks - ok, there is no file because Duplicator has not run at least one time. Maybe the PHP script is killed abruptly. Please check your PM.

 

edit:

I didn't saw the last error message, there is something broken with the path.

[theo]

Yes, and I can tell you that it works, when I configure a virtual host in Apache and add "127.0.0.1    pwdev.ch" in the hosts file.

<VirtualHost *:80>
    ServerAdmin some@some.ch
    ServerAlias pwdev.ch
    ServerName www.pwdev.ch
    DocumentRoot /home/theo/public_html/pwdev
</VirtualHost>

Then running under http://pwdev.ch/processwire/ everything is fine. 

So it has to do with mod_userdir.

Thank you.

[flydev]

Thanks to you for reporting it back. I will test it on a server with mod_userdir installed today.

[jacmaes]

Thanks @flydev for your work on this module. I keep getting the following error when trying to use FTP:

"[FTP] error: invalid file 2017-12-04_07-01-36-glosas.anle.us.package.zip, bad extension"

I don't understand what "bad extension" means. I've tried leaving the Name field for the backup file blank (default state), and changing it also (which by the way seems to have no effect), but I get the same error.

Another question: are you going to upload the Google Drive SDK? I get a 404 error when clicking on "you can download it here..."

Thanks.

[flydev]

14 minutes ago, jacmaes said:

I don't understand what "bad extension" means.

Never saw this issue with a FTP server, maybe its a specific settings of the server ?

Do you have good results uploading a conventional file, like a "foobar.txt", an image or even a  zip file ?

I think I found the issue, it has nothing to do with server, I am dumb. I will push a fix.

 

14 minutes ago, jacmaes said:

are you going to upload the Google Drive SDK?

If I have the time, i will upload the SDKs tonight - I have to update and test them before.

 

---

 

56 minutes ago, theo said:

So it has to do with mod_userdir.

I can confirm this issue as I could reproduce it right now, I will find a fix.

[flydev]

@jacmaes Could you test the last version (1.1.3) please ?

[flydev]

@theo :  I just pushed a fix - now Duplicator work with Apache mod_userdir.

If you can upgrade the module to the latest version from Github or the module directory (version 1.1.4) and report back, I will be happy

 

 

edit:  don't forget to change the const DUP_MIN_VERSION

[theo]

8 minutes ago, flydev said:

@theo :  I just pushed a fix - now Duplicator work with Apache mod_userdir.

If you can upgrade the module to the latest version from Github or the module directory and report back, I will be happy

 

Thank you.

It works now with mod_userdir and PW 3.0.83, but it does still not work with 2.7.2

The log says:

2017-12-07 15:23:13:  Package build failed.
2017-12-07 15:23:13:  An error occured during package build.
2017-12-07 15:23:13:  An error occured while building the ProcessWire structure.

 

[flydev]

I just tested Duplicator on 2.7.2 fresh install, everything is fine.

Testing with mod_userdir... I will edit this post.

 

Edit: @theo  yep it work on my side with mod_userdir, ProcessWire 2.7.2, Duplicator 1.1.4

Maybe removing the cache and/or a modules refresh could fix it.

[theo]

20 minutes ago, flydev said:

I just tested Duplicator on 2.7.2 fresh install, everything is fine.

Testing with mod_userdir... I will edit this post.

 

Edit: @theo  yep it work on my side with mod_userdir, ProcessWire 2.7.2, Duplicator 1.1.4

Maybe removing the cache and/or a modules refresh could fix it.

Thank you, but no, same problem.

Maybe there is something "wrong" with my installation.