flydev

Duplicator: Backup and move sites

Recommended Posts

I will make Duplicator more verbose about the logs in the next version.

Thanks again for your time. Stay tuned :)

  • Like 1

Share this post


Link to post
Share on other sites

Regarding mod_userdir - I am curious about this - is this going to affect other PW modules, or the core even - it looks like might? Can you guys let us know what $this->config->urls->admin . 'setup/' returns when mod_userdir is installed?

FYI - with that recent fix, you can avoid that str_replace by going for the option on line #3

5a295df7cb971_ScreenShot2017-12-07at7_26_35AM.png.fd353f765c0ebd7ae1a33555837297b1.png

  • Like 1

Share this post


Link to post
Share on other sites
10 minutes ago, adrian said:

Regarding mod_userdir - I am curious about this - is this going to affect other PW modules, or the core even - it looks like might? Can you guys let us know what $this->config->urls->admin . 'setup/' returns when mod_userdir is installed?

 

 

Here it returns:

/~theo/pwdev/processwire/setup/

 

Thanks.

  • Like 1

Share this post


Link to post
Share on other sites
16 minutes ago, adrian said:

Can you guys let us know what $this->config->urls->admin . 'setup/' returns when mod_userdir is installed?

I was going to post it as the result is weird normal..., but didn't knew about httpAdmin

I am so tired...

 

debug2.thumb.png.66e9252431350244708b9ba341dfb75e.png

  • Like 1

Share this post


Link to post
Share on other sites
10 minutes ago, flydev said:

I was going to post it as the result is weird normal..., but didn't knew about httpAdmin

I guess I don't actually understand why $this->config->urls->admin . 'setup/' isn't working. It seems like a valid url and I don't understand why the full http:// url works.

As for httpAdmin - any of the config->url options can have the http prefix and they will work - very handy!

  • Thanks 1

Share this post


Link to post
Share on other sites
47 minutes ago, adrian said:

I guess I don't actually understand why $this->config->urls->admin . 'setup/' isn't working. It seems like a valid url and I don't understand why the full http:// url works.

I don't know what it is supposed to return, but

/~theo/pwdev/processwire/setup/

is not a valid path on the file system.

I think confusion comes from the "calculation" of file paths.

Share this post


Link to post
Share on other sites

@flydev - just noticed the order of local vs Google Drive packages is reversed. Would be nice to have them both in descending order. Should they maybe also blended together?

5a29737ec142d_ScreenShot2017-12-07at8_58_32AM.png.bb56cd8d12593803411f7ddcc698f383.png

  • Like 1

Share this post


Link to post
Share on other sites
30 minutes ago, theo said:

is not a valid path on the file system.

I think confusion comes from the "calculation" of file paths.

It return an URL, not the path ;)  In this case, the URL of your admin page which is valid.

 

@adrian I think I already tried but not in depth to blend them.  I will look at it more closely, its definitely a good idea.

Edited by flydev
precision
  • Like 2

Share this post


Link to post
Share on other sites
24 minutes ago, flydev said:

It return an URL, not the path ;)  In this case, the URL of your admin page which is valid.

Yes sure, but still adrian was asking how mod_userdir may affect other modules.

Where normally

http://mydomain.com/processwire

maps to

/myfilesystemroot/processwire

http://localhost/~theo/pwdev/processwire/

does not follow the same rule:

/myfilesystemroot/~theo/pwdev/processwire/

is not a valid file path.

This should be:

/myfilesystemroot/pwdev/processwire/

  • Like 2

Share this post


Link to post
Share on other sites

Hmm, there is a little regression on a free webhosting platform I am using:

Fatal error: Call to undefined function apache_get_modules() in /users/pwdev/www/site/assets/cache/FileCompiler/site/modules/Duplicator/ProcessDuplicator.module on line 250

The module worked before on this host, but not any longer in version 1.1.4

http://phpinfo.square7.ch/

Thank you.

  • Thanks 1

Share this post


Link to post
Share on other sites
5 minutes ago, theo said:

Hmm, there is a little regression on a free webhosting platform I am using:


Fatal error: Call to undefined function apache_get_modules() in /users/pwdev/www/site/assets/cache/FileCompiler/site/modules/Duplicator/ProcessDuplicator.module on line 250

The module worked before on this host, but not any longer in version 1.1.4

http://phpinfo.square7.ch/

Thank you.

@flydev - this might save you some time:
 https://github.com/adrianbj/TracyDebugger/blob/c6335760db5467fe018a30c853abf7d4c1c6c967/panels/ProcesswireInfoPanel.php#L288-L306

It provides other options for detecting mod_rewrite, but might also be helpful for mod_userdir

  • Like 2

Share this post


Link to post
Share on other sites
3 minutes ago, adrian said:

@flydev - this might save you some time:
 https://github.com/adrianbj/TracyDebugger/blob/c6335760db5467fe018a30c853abf7d4c1c6c967/panels/ProcesswireInfoPanel.php#L288-L306

It provides other options for detecting mod_rewrite, but might also be helpful for mod_userdir

Thanks, I will read that.

I have solved the problem with (Line 250)

if (function_exists('apache_get_modules') && (in_array('mod_userdir', apache_get_modules()))) {

 

Share this post


Link to post
Share on other sites

Thanks for the code snippet @adrian ,  I think that checking for mod_userdir is not needed anymore as the only issue occurring with this apache module was getting the backend url, which is fixed by using $config->urls->httpAdmin

 

18 minutes ago, theo said:

The module worked before on this host, but not any longer in version 1.1.4

I will push the update in an hour or so. Thanks!

  • Like 3

Share this post


Link to post
Share on other sites
4 minutes ago, flydev said:

Thanks for the code snippet @adrian ,  I think that checking for mod_userdir is not needed anymore as the only issue occurring with this apache module was getting the backend url, which is fixed by using $config->urls->httpAdmin

 

I will push the update in an hour or so. Thanks!

Great. I didn't really understand what it is doing anyway.

The fact that mod_userdir is installed, does not mean, that it is actually used.

Thank you.

  • Like 1

Share this post


Link to post
Share on other sites

I am trying to use the duplicator module and I can't get it to create a 'package'. I can click on the button to create one but it just shows "processing..." briefly and then goes back to the create package button. 

Does anyone have experience with this module?

 

Matt

Edited by adrian
Merged into the correct thread for this module

Share this post


Link to post
Share on other sites

The module is updated :

- fixed ProcessDuplicator

- packages listed in ProcessDuplicator are now blended and re-ordered correctly by timestamps

blendedpackages.png.d235282ab5c649085e6293c3982f2cf1.png

 

 

 

@msavard Hi and welcome here!

About your issue, it is hard say without furthermore information. It can be a memory issue or a timeout issue.

Which version of ProcessWire / PHP are you trying to run Duplicator on ?

Which hosting provider are you using ? Are you on vps, shared host ?

Could you check the duplicator log (in /site/assets/logs/duplicator.log), the server and PHP error logs ?

 

 

 

 

  • Like 3
  • Thanks 1

Share this post


Link to post
Share on other sites

I am running ProcessWire 3.0.77, PHP v5.6.30

There is no duplicator log in the logs folder. I didn't see any errors on the server or PHP logs.

I am running XAMP on my local machine for this test site.

I do not have any CRON or timings settings defined but have selected none. I assumed this mean I could run the process manually.

Share this post


Link to post
Share on other sites

I just installed a fresh XAMP on a Windows machine for a test and everything worked there. (XAMP and the bitnami PW module)

 

12 hours ago, msavard said:

I can click on the button to create one but it just shows "processing..." briefly and then goes back to the create package button. 

Are you running the last version of Duplicator (1.1.5) ?

When you say briefly, did you mean "instantly" ? if yes,  it can be the problem that @theo encountered which is fixed in the 1.1.5.

Please let me know, thanks you.

  • Like 1

Share this post


Link to post
Share on other sites

Hi flydev

Thank you for your patience. ;-)

I have still the same problem with my 2.7.2.

After pressing the button "Initiate Backup Process" it says "Processing..." then it seems to do something for some 10 seconds and then it returns to "Initiate Backup Process".

The log says:

2017-12-08 10:42:15:  Package build failed.
2017-12-08 10:42:15:  An error occured during package build.
2017-12-08 10:42:15:  An error occured while building the ProcessWire structure.

Where could I try to find the problem?

P.S. I have ChromePHPLogger installed.

The image below for example is the result of

ChromePhp::log($zipinfo);
ChromePhp::log($fileinfo);

in Line 311 of Duplicator Module. Does it mean anything?

duplicator1.png

  • Like 1

Share this post


Link to post
Share on other sites

Hi @theo

If TracyDebugger is not installed, then install it and re-run Duplicator to see if the debugger catch an error or warning.

Then check the PHP error log to if there is something wrong.

And If you got a bit time, then you can try to debug the buildDatabaseBackup(), buildProcessWireBackup and buildPackage() functions with putting tracy's shortcut bd($your_var) everywhere ;)

Or just wait that I implement more logging in the module (I will try to find the time today).

 

Edit: looking at the screenshot, it look like buildProcessWireBackup() run fine.

Share this post


Link to post
Share on other sites
19 minutes ago, flydev said:

Edit: looking at the screenshot, it look like buildProcessWireBackup() run fine.

But it doesn't.

Next line after the debug log commands above is

       if (!file_exists($fileinfo['zipfile'])) {
            DUP_Logs::log("An error occured while building the ProcessWire structure.");
            return false;
        }

And this throws the message "An error occured while building the ProcessWire structure."

so zipfile does not exist and buildProcessWireBackup returns false.

Share this post


Link to post
Share on other sites

I can't find the problem. Here again the logs of the variables in buildProcessWireBackup:

log.js:137 $root:
log.js:137 /home/theo/public_html/mysite/
log.js:137 $path:
log.js:137 /home/theo/public_html/mysite/site/assets/
log.js:137 $options:
log.js:137 {filename: "2017-12-08_11-56-07-localhost.pw.zip", folder: {…}, exclude: Array(8), extension: Array(0)}exclude: Array(8)
0: "/home/theo/public_html/mysite/site/assets/cache/WireTempDir"
1: "/home/theo/public_html/mysite/wire"
2: "/home/theo/public_html/mysite/site/assets/backups"
3: "/home/theo/public_html/mysite/site/assets/logs"
4: "/home/theo/public_html/mysite/site/assets/cache"
5: "/home/theo/public_html/mysite/site/assets/sessions"
6: "/home/theo/public_html/mysite/site/assets/files"
7: ""
length: 8__proto__: Array(0)
extension: []length: 0__proto__: Array(0)
filename: "2017-12-08_11-56-07-localhost.pw.zip"
folder: {}__proto__: Object__proto__: Object
log.js:137 $zipinfo:
log.js:137 {files: 4951, dirs: 0, size: 280395773}dirs: 0files: 4951size: 280395773__proto__: Object
log.js:137 $fileinfo:
log.js:137 {zipfile: "/home/theo/public_html/mysite/site/assets/cach…cator-temp/1/2017-12-08_11-56-07-localhost.pw.zip", zipname: "2017-12-08_11-56-07-localhost.pw.zip", structure: {…}}structure: {files: 4951, dirs: 0, size: 280395773}zipfile: "/home/theo/public_html/mysite/site/assets/cache/WireTempDir/.duplicator-temp/1/2017-12-08_11-56-07-localhost.pw.zip"zipname: "2017-12-08_11-56-07-localhost.pw.zip"__proto__: Object

Thank you

  • Thanks 1

Share this post


Link to post
Share on other sites

Strange thing in function zipData

I add these logs around $zip->close()

    ChromePhp::log($zip);
    ChromePhp::log($zip->close());
    ChromePhp::log($zip);
log.js:137 ZipArchive {status: 0, statusSys: 0, numFiles: 4951, filename: "/home/theo/public_html/mysite/site/assets/cach…cator-temp/1/2017-12-08_13-42-33-localhost.pw.zip", comment: ""}
log.js:137 false
log.js:137 ZipArchive {status: 0, statusSys: 0, numFiles: 0, filename: "", comment: ""}

 

$zip->close() returns false.

What could be the reason?

Share this post


Link to post
Share on other sites

OK, some success here: Please add "is_readable" check  in function zipData

 } elseif (is_file($file) && is_readable($file)) {
                            $filecount++;
                            $zip->addFile($file, str_replace($source . '', '', $file));
                            $limitItems++;

Like this it works. If a file is not readable, it silently fails otherwise.

  • Like 3

Share this post


Link to post
Share on other sites

Answering from my mobile.

Good catch ! I will update the module with  your patch, we need now to alert the user that some files are not resdable and thus excluded from the zip. Thanks again @theo!

  • Like 3

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By holmescreek
      Question 1
      I recently installed PW 3.0.62 for a new site and also have sites running older version of PW 3.x.
      Can I export the database on an older version of PW 3.x and import it to PW 3.0.62 without any issues?
       
      Question 2
      (This is kind of alternative to the above for long term use - and maybe a better solution...)
      On  the sites I've previously built I have templates (home, basic-page, contact) and fields that I commonly use, such as business_name, phone_1.  The last site I built is running PW 3.0.42.  I was considering cloning this into a local site and running the upgrade module to bring it up to PW 3.0.62. From there on out when I start I new project I could just run the PW upgrade module, copy the folder to the location for the new project and duplicate the database using the new projects name.

      So basically, I'll always keep a "blank slate" site that I can just run the PW upgrade on, then duplicate into a new project. This would cut down on the work and time spent having to re-create these common fields, that I use. From there, I would just add fields, templates, etc, specific for the new website project.

      Is this a sound approach to speed up development?

       
    • By holmescreek
      I just posted a question about moving PW from a resting domain to a production domain, after consideration I figure I would create a new post for a second question I have.
      I noticed the coming feature of client size image resizing. This will be really helpful, as I have some clients on a Satellite connection. I'm telling you, them trying to upload 15MB images, or them getting them straight on batch resizers (or Photoshop) to resize before upload is a real chore. Needless to say, Ryan has been doing an excellent job addressing this.
      My question, is that I have several clients on PW 2.x thru 3.x. Is there a really good guide on steps that I need to take to update the core to take advantage of the features of new releases? Are their perils to look out for?
      Yeah, after developing several sites, I feel a bit silly about asking this one, but, the docs doesn't seem to be clear in a "general" sense of upgrading the core across all versions.
      Thanks everyone for your time and assistance.
    • By suntrop
      Hi all
      I have big troubles with my DB backups and phpMyAdmin. I can't manage to import a backup.
      The phpMyAdmin export settings are left to the defaults. The error message I get:
      #1064 - You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'INSERT INTO `caches` (`name`, `data`, `expires`) VALUES ('ModulesUninstalled.in' at line 11 When I delete the complete 'cache' section there are almost for all tables the same errors. (part of the dump attached)
      I had this problem before and back then (couple months ago) I think I copied almost every table individually. I thought it was specific to that DB. But I can't do a dump/import of any of my PW installations.
      MySQL: 5.6.27
      PHP version: 7.0.11
      phpMyAdmin: 4.5.5.1
      Anybody knows that problem? Or what to do?
      phpMyAdmin SQL Dump.sql
    • By vwatson
      Is it possible to add check boxes in order to select multiple backup files and delete them all at once, rather than having to click each one, then Delete?
      Also it would be helpful to make the Restore process more distinct from Deleting.  I must have inadvertently clicked the wrong button once and restored a backup unintentionally, wiping out a day's work. My mistake of course. I obviously wasn't paying close enough attention.  
    • By Fantomas
      Hi all,
      I'm still new to processwire and now I have to create the first hook.
      I need to modify the page-clone module, so there is a field in pages that has to be empty after cloning the page. How could I start this task?