☁️ Duplicator: Backup and move sites

[flydev]

I will make Duplicator more verbose about the logs in the next version.

Thanks again for your time. Stay tuned

[adrian]

Regarding mod_userdir - I am curious about this - is this going to affect other PW modules, or the core even - it looks like might? Can you guys let us know what $this->config->urls->admin . 'setup/' returns when mod_userdir is installed?

FYI - with that recent fix, you can avoid that str_replace by going for the option on line #3

[theo]

10 minutes ago, adrian said:

Regarding mod_userdir - I am curious about this - is this going to affect other PW modules, or the core even - it looks like might? Can you guys let us know what $this->config->urls->admin . 'setup/' returns when mod_userdir is installed?

 

 

Here it returns:

/~theo/pwdev/processwire/setup/

 

Thanks.

[flydev]

16 minutes ago, adrian said:

Can you guys let us know what $this->config->urls->admin . 'setup/' returns when mod_userdir is installed?

I was going to post it as the result is weird normal..., but didn't knew about httpAdmin ! 

I am so tired...

 

[adrian]

10 minutes ago, flydev said:

I was going to post it as the result is weird normal..., but didn't knew about httpAdmin ! 

I guess I don't actually understand why $this->config->urls->admin . 'setup/' isn't working. It seems like a valid url and I don't understand why the full http:// url works.

As for httpAdmin - any of the config->url options can have the http prefix and they will work - very handy!

[theo]

47 minutes ago, adrian said:

I guess I don't actually understand why $this->config->urls->admin . 'setup/' isn't working. It seems like a valid url and I don't understand why the full http:// url works.

I don't know what it is supposed to return, but

/~theo/pwdev/processwire/setup/

is not a valid path on the file system.

I think confusion comes from the "calculation" of file paths.

[adrian]

@flydev - just noticed the order of local vs Google Drive packages is reversed. Would be nice to have them both in descending order. Should they maybe also blended together?

[flydev]

30 minutes ago, theo said:

is not a valid path on the file system.

I think confusion comes from the "calculation" of file paths.

It return an URL, not the path   In this case, the URL of your admin page which is valid.

 

@adrian I think I already tried but not in depth to blend them.  I will look at it more closely, its definitely a good idea.

Edited December 7, 2017 by flydev
precision

[theo]

24 minutes ago, flydev said:

It return an URL, not the path   In this case, the URL of your admin page which is valid.

Yes sure, but still adrian was asking how mod_userdir may affect other modules.

Where normally

http://mydomain.com/processwire

maps to

/myfilesystemroot/processwire

http://localhost/~theo/pwdev/processwire/

does not follow the same rule:

/myfilesystemroot/~theo/pwdev/processwire/

is not a valid file path.

This should be:

/myfilesystemroot/pwdev/processwire/

[theo]

Hmm, there is a little regression on a free webhosting platform I am using:

Fatal error: Call to undefined function apache_get_modules() in /users/pwdev/www/site/assets/cache/FileCompiler/site/modules/Duplicator/ProcessDuplicator.module on line 250

The module worked before on this host, but not any longer in version 1.1.4

http://phpinfo.square7.ch/

Thank you.

[adrian]

5 minutes ago, theo said:

Hmm, there is a little regression on a free webhosting platform I am using:

Fatal error: Call to undefined function apache_get_modules() in /users/pwdev/www/site/assets/cache/FileCompiler/site/modules/Duplicator/ProcessDuplicator.module on line 250

The module worked before on this host, but not any longer in version 1.1.4

http://phpinfo.square7.ch/

Thank you.

@flydev - this might save you some time:
 https://github.com/adrianbj/TracyDebugger/blob/c6335760db5467fe018a30c853abf7d4c1c6c967/panels/ProcesswireInfoPanel.php#L288-L306

It provides other options for detecting mod_rewrite, but might also be helpful for mod_userdir

[theo]

3 minutes ago, adrian said:

@flydev - this might save you some time:
 https://github.com/adrianbj/TracyDebugger/blob/c6335760db5467fe018a30c853abf7d4c1c6c967/panels/ProcesswireInfoPanel.php#L288-L306

It provides other options for detecting mod_rewrite, but might also be helpful for mod_userdir

Thanks, I will read that.

I have solved the problem with (Line 250)

if (function_exists('apache_get_modules') && (in_array('mod_userdir', apache_get_modules()))) {

 

[flydev]

Thanks for the code snippet @adrian ,  I think that checking for mod_userdir is not needed anymore as the only issue occurring with this apache module was getting the backend url, which is fixed by using $config->urls->httpAdmin

 

18 minutes ago, theo said:

The module worked before on this host, but not any longer in version 1.1.4

I will push the update in an hour or so. Thanks!

[theo]

4 minutes ago, flydev said:

Thanks for the code snippet @adrian ,  I think that checking for mod_userdir is not needed anymore as the only issue occurring with this apache module was getting the backend url, which is fixed by using $config->urls->httpAdmin

 

I will push the update in an hour or so. Thanks!

Great. I didn't really understand what it is doing anyway.

The fact that mod_userdir is installed, does not mean, that it is actually used.

Thank you.

[msavard]

I am trying to use the duplicator module and I can't get it to create a 'package'. I can click on the button to create one but it just shows "processing..." briefly and then goes back to the create package button. 

Does anyone have experience with this module?

 

Matt

Edited December 7, 2017 by adrian
Merged into the correct thread for this module

[flydev]

The module is updated :

- fixed ProcessDuplicator

- packages listed in ProcessDuplicator are now blended and re-ordered correctly by timestamps

 

 

 

@msavard Hi and welcome here!

About your issue, it is hard say without furthermore information. It can be a memory issue or a timeout issue.

Which version of ProcessWire / PHP are you trying to run Duplicator on ?

Which hosting provider are you using ? Are you on vps, shared host ?

Could you check the duplicator log (in /site/assets/logs/duplicator.log), the server and PHP error logs ?

 

 

 

 

[msavard]

I am running ProcessWire 3.0.77, PHP v5.6.30

There is no duplicator log in the logs folder. I didn't see any errors on the server or PHP logs.

I am running XAMP on my local machine for this test site.

I do not have any CRON or timings settings defined but have selected none. I assumed this mean I could run the process manually.

[flydev]

I just installed a fresh XAMP on a Windows machine for a test and everything worked there. (XAMP and the bitnami PW module)

 

12 hours ago, msavard said:

I can click on the button to create one but it just shows "processing..." briefly and then goes back to the create package button. 

Are you running the last version of Duplicator (1.1.5) ?

When you say briefly, did you mean "instantly" ? if yes,  it can be the problem that @theo encountered which is fixed in the 1.1.5.

Please let me know, thanks you.

[theo]

Hi flydev

Thank you for your patience. ;-)

I have still the same problem with my 2.7.2.

After pressing the button "Initiate Backup Process" it says "Processing..." then it seems to do something for some 10 seconds and then it returns to "Initiate Backup Process".

The log says:

2017-12-08 10:42:15:  Package build failed.
2017-12-08 10:42:15:  An error occured during package build.
2017-12-08 10:42:15:  An error occured while building the ProcessWire structure.

Where could I try to find the problem?

P.S. I have ChromePHPLogger installed.

The image below for example is the result of

ChromePhp::log($zipinfo);
ChromePhp::log($fileinfo);

in Line 311 of Duplicator Module. Does it mean anything?

[flydev]

Hi @theo

If TracyDebugger is not installed, then install it and re-run Duplicator to see if the debugger catch an error or warning.

Then check the PHP error log to if there is something wrong.

And If you got a bit time, then you can try to debug the buildDatabaseBackup(), buildProcessWireBackup and buildPackage() functions with putting tracy's shortcut bd($your_var) everywhere

Or just wait that I implement more logging in the module (I will try to find the time today).

 

Edit: looking at the screenshot, it look like buildProcessWireBackup() run fine.

[theo]

19 minutes ago, flydev said:

Edit: looking at the screenshot, it look like buildProcessWireBackup() run fine.

But it doesn't.

Next line after the debug log commands above is

       if (!file_exists($fileinfo['zipfile'])) {
            DUP_Logs::log("An error occured while building the ProcessWire structure.");
            return false;
        }

And this throws the message "An error occured while building the ProcessWire structure."

so zipfile does not exist and buildProcessWireBackup returns false.

[theo]

I can't find the problem. Here again the logs of the variables in buildProcessWireBackup:

log.js:137 $root:
log.js:137 /home/theo/public_html/mysite/
log.js:137 $path:
log.js:137 /home/theo/public_html/mysite/site/assets/
log.js:137 $options:
log.js:137 {filename: "2017-12-08_11-56-07-localhost.pw.zip", folder: {…}, exclude: Array(8), extension: Array(0)}exclude: Array(8)
0: "/home/theo/public_html/mysite/site/assets/cache/WireTempDir"
1: "/home/theo/public_html/mysite/wire"
2: "/home/theo/public_html/mysite/site/assets/backups"
3: "/home/theo/public_html/mysite/site/assets/logs"
4: "/home/theo/public_html/mysite/site/assets/cache"
5: "/home/theo/public_html/mysite/site/assets/sessions"
6: "/home/theo/public_html/mysite/site/assets/files"
7: ""
length: 8__proto__: Array(0)
extension: []length: 0__proto__: Array(0)
filename: "2017-12-08_11-56-07-localhost.pw.zip"
folder: {}__proto__: Object__proto__: Object
log.js:137 $zipinfo:
log.js:137 {files: 4951, dirs: 0, size: 280395773}dirs: 0files: 4951size: 280395773__proto__: Object
log.js:137 $fileinfo:
log.js:137 {zipfile: "/home/theo/public_html/mysite/site/assets/cach…cator-temp/1/2017-12-08_11-56-07-localhost.pw.zip", zipname: "2017-12-08_11-56-07-localhost.pw.zip", structure: {…}}structure: {files: 4951, dirs: 0, size: 280395773}zipfile: "/home/theo/public_html/mysite/site/assets/cache/WireTempDir/.duplicator-temp/1/2017-12-08_11-56-07-localhost.pw.zip"zipname: "2017-12-08_11-56-07-localhost.pw.zip"__proto__: Object

Thank you

[theo]

Strange thing in function zipData

I add these logs around $zip->close()

    ChromePhp::log($zip);
    ChromePhp::log($zip->close());
    ChromePhp::log($zip);

log.js:137 ZipArchive {status: 0, statusSys: 0, numFiles: 4951, filename: "/home/theo/public_html/mysite/site/assets/cach…cator-temp/1/2017-12-08_13-42-33-localhost.pw.zip", comment: ""}
log.js:137 false
log.js:137 ZipArchive {status: 0, statusSys: 0, numFiles: 0, filename: "", comment: ""}

 

$zip->close() returns false.

What could be the reason?

[theo]

OK, some success here: Please add "is_readable" check  in function zipData

 } elseif (is_file($file) && is_readable($file)) {
                            $filecount++;
                            $zip->addFile($file, str_replace($source . '', '', $file));
                            $limitItems++;

Like this it works. If a file is not readable, it silently fails otherwise.

[flydev]

Answering from my mobile.

Good catch ! I will update the module with  your patch, we need now to alert the user that some files are not resdable and thus excluded from the zip. Thanks again @theo!