Jump to content
flydev

Duplicator: Backup and move sites

Recommended Posts

I will make Duplicator more verbose about the logs in the next version.

Thanks again for your time. Stay tuned :)

  • Like 1

Share this post


Link to post
Share on other sites

Regarding mod_userdir - I am curious about this - is this going to affect other PW modules, or the core even - it looks like might? Can you guys let us know what $this->config->urls->admin . 'setup/' returns when mod_userdir is installed?

FYI - with that recent fix, you can avoid that str_replace by going for the option on line #3

5a295df7cb971_ScreenShot2017-12-07at7_26_35AM.png.fd353f765c0ebd7ae1a33555837297b1.png

  • Like 1

Share this post


Link to post
Share on other sites
10 minutes ago, adrian said:

Regarding mod_userdir - I am curious about this - is this going to affect other PW modules, or the core even - it looks like might? Can you guys let us know what $this->config->urls->admin . 'setup/' returns when mod_userdir is installed?

 

 

Here it returns:

/~theo/pwdev/processwire/setup/

 

Thanks.

  • Like 1

Share this post


Link to post
Share on other sites
16 minutes ago, adrian said:

Can you guys let us know what $this->config->urls->admin . 'setup/' returns when mod_userdir is installed?

I was going to post it as the result is weird normal..., but didn't knew about httpAdmin

I am so tired...

 

debug2.thumb.png.66e9252431350244708b9ba341dfb75e.png

  • Like 1

Share this post


Link to post
Share on other sites
10 minutes ago, flydev said:

I was going to post it as the result is weird normal..., but didn't knew about httpAdmin

I guess I don't actually understand why $this->config->urls->admin . 'setup/' isn't working. It seems like a valid url and I don't understand why the full http:// url works.

As for httpAdmin - any of the config->url options can have the http prefix and they will work - very handy!

  • Thanks 1

Share this post


Link to post
Share on other sites
47 minutes ago, adrian said:

I guess I don't actually understand why $this->config->urls->admin . 'setup/' isn't working. It seems like a valid url and I don't understand why the full http:// url works.

I don't know what it is supposed to return, but

/~theo/pwdev/processwire/setup/

is not a valid path on the file system.

I think confusion comes from the "calculation" of file paths.

Share this post


Link to post
Share on other sites

@flydev - just noticed the order of local vs Google Drive packages is reversed. Would be nice to have them both in descending order. Should they maybe also blended together?

5a29737ec142d_ScreenShot2017-12-07at8_58_32AM.png.bb56cd8d12593803411f7ddcc698f383.png

  • Like 1

Share this post


Link to post
Share on other sites
30 minutes ago, theo said:

is not a valid path on the file system.

I think confusion comes from the "calculation" of file paths.

It return an URL, not the path ;)  In this case, the URL of your admin page which is valid.

 

@adrian I think I already tried but not in depth to blend them.  I will look at it more closely, its definitely a good idea.

Edited by flydev
precision
  • Like 2

Share this post


Link to post
Share on other sites
24 minutes ago, flydev said:

It return an URL, not the path ;)  In this case, the URL of your admin page which is valid.

Yes sure, but still adrian was asking how mod_userdir may affect other modules.

Where normally

http://mydomain.com/processwire

maps to

/myfilesystemroot/processwire

http://localhost/~theo/pwdev/processwire/

does not follow the same rule:

/myfilesystemroot/~theo/pwdev/processwire/

is not a valid file path.

This should be:

/myfilesystemroot/pwdev/processwire/

  • Like 2

Share this post


Link to post
Share on other sites

Hmm, there is a little regression on a free webhosting platform I am using:

Fatal error: Call to undefined function apache_get_modules() in /users/pwdev/www/site/assets/cache/FileCompiler/site/modules/Duplicator/ProcessDuplicator.module on line 250

The module worked before on this host, but not any longer in version 1.1.4

http://phpinfo.square7.ch/

Thank you.

  • Thanks 1

Share this post


Link to post
Share on other sites
5 minutes ago, theo said:

Hmm, there is a little regression on a free webhosting platform I am using:


Fatal error: Call to undefined function apache_get_modules() in /users/pwdev/www/site/assets/cache/FileCompiler/site/modules/Duplicator/ProcessDuplicator.module on line 250

The module worked before on this host, but not any longer in version 1.1.4

http://phpinfo.square7.ch/

Thank you.

@flydev - this might save you some time:
 https://github.com/adrianbj/TracyDebugger/blob/c6335760db5467fe018a30c853abf7d4c1c6c967/panels/ProcesswireInfoPanel.php#L288-L306

It provides other options for detecting mod_rewrite, but might also be helpful for mod_userdir

  • Like 2

Share this post


Link to post
Share on other sites
3 minutes ago, adrian said:

@flydev - this might save you some time:
 https://github.com/adrianbj/TracyDebugger/blob/c6335760db5467fe018a30c853abf7d4c1c6c967/panels/ProcesswireInfoPanel.php#L288-L306

It provides other options for detecting mod_rewrite, but might also be helpful for mod_userdir

Thanks, I will read that.

I have solved the problem with (Line 250)

if (function_exists('apache_get_modules') && (in_array('mod_userdir', apache_get_modules()))) {

 

Share this post


Link to post
Share on other sites

Thanks for the code snippet @adrian ,  I think that checking for mod_userdir is not needed anymore as the only issue occurring with this apache module was getting the backend url, which is fixed by using $config->urls->httpAdmin

 

18 minutes ago, theo said:

The module worked before on this host, but not any longer in version 1.1.4

I will push the update in an hour or so. Thanks!

  • Like 3

Share this post


Link to post
Share on other sites
4 minutes ago, flydev said:

Thanks for the code snippet @adrian ,  I think that checking for mod_userdir is not needed anymore as the only issue occurring with this apache module was getting the backend url, which is fixed by using $config->urls->httpAdmin

 

I will push the update in an hour or so. Thanks!

Great. I didn't really understand what it is doing anyway.

The fact that mod_userdir is installed, does not mean, that it is actually used.

Thank you.

  • Like 1

Share this post


Link to post
Share on other sites

I am trying to use the duplicator module and I can't get it to create a 'package'. I can click on the button to create one but it just shows "processing..." briefly and then goes back to the create package button. 

Does anyone have experience with this module?

 

Matt

Edited by adrian
Merged into the correct thread for this module

Share this post


Link to post
Share on other sites

The module is updated :

- fixed ProcessDuplicator

- packages listed in ProcessDuplicator are now blended and re-ordered correctly by timestamps

blendedpackages.png.d235282ab5c649085e6293c3982f2cf1.png

 

 

 

@msavard Hi and welcome here!

About your issue, it is hard say without furthermore information. It can be a memory issue or a timeout issue.

Which version of ProcessWire / PHP are you trying to run Duplicator on ?

Which hosting provider are you using ? Are you on vps, shared host ?

Could you check the duplicator log (in /site/assets/logs/duplicator.log), the server and PHP error logs ?

 

 

 

 

  • Like 3
  • Thanks 1

Share this post


Link to post
Share on other sites

I am running ProcessWire 3.0.77, PHP v5.6.30

There is no duplicator log in the logs folder. I didn't see any errors on the server or PHP logs.

I am running XAMP on my local machine for this test site.

I do not have any CRON or timings settings defined but have selected none. I assumed this mean I could run the process manually.

Share this post


Link to post
Share on other sites

I just installed a fresh XAMP on a Windows machine for a test and everything worked there. (XAMP and the bitnami PW module)

 

12 hours ago, msavard said:

I can click on the button to create one but it just shows "processing..." briefly and then goes back to the create package button. 

Are you running the last version of Duplicator (1.1.5) ?

When you say briefly, did you mean "instantly" ? if yes,  it can be the problem that @theo encountered which is fixed in the 1.1.5.

Please let me know, thanks you.

  • Like 1

Share this post


Link to post
Share on other sites

Hi flydev

Thank you for your patience. ;-)

I have still the same problem with my 2.7.2.

After pressing the button "Initiate Backup Process" it says "Processing..." then it seems to do something for some 10 seconds and then it returns to "Initiate Backup Process".

The log says:

2017-12-08 10:42:15:  Package build failed.
2017-12-08 10:42:15:  An error occured during package build.
2017-12-08 10:42:15:  An error occured while building the ProcessWire structure.

Where could I try to find the problem?

P.S. I have ChromePHPLogger installed.

The image below for example is the result of

ChromePhp::log($zipinfo);
ChromePhp::log($fileinfo);

in Line 311 of Duplicator Module. Does it mean anything?

duplicator1.png

  • Like 1

Share this post


Link to post
Share on other sites

Hi @theo

If TracyDebugger is not installed, then install it and re-run Duplicator to see if the debugger catch an error or warning.

Then check the PHP error log to if there is something wrong.

And If you got a bit time, then you can try to debug the buildDatabaseBackup(), buildProcessWireBackup and buildPackage() functions with putting tracy's shortcut bd($your_var) everywhere ;)

Or just wait that I implement more logging in the module (I will try to find the time today).

 

Edit: looking at the screenshot, it look like buildProcessWireBackup() run fine.

Share this post


Link to post
Share on other sites
19 minutes ago, flydev said:

Edit: looking at the screenshot, it look like buildProcessWireBackup() run fine.

But it doesn't.

Next line after the debug log commands above is

       if (!file_exists($fileinfo['zipfile'])) {
            DUP_Logs::log("An error occured while building the ProcessWire structure.");
            return false;
        }

And this throws the message "An error occured while building the ProcessWire structure."

so zipfile does not exist and buildProcessWireBackup returns false.

Share this post


Link to post
Share on other sites

I can't find the problem. Here again the logs of the variables in buildProcessWireBackup:

log.js:137 $root:
log.js:137 /home/theo/public_html/mysite/
log.js:137 $path:
log.js:137 /home/theo/public_html/mysite/site/assets/
log.js:137 $options:
log.js:137 {filename: "2017-12-08_11-56-07-localhost.pw.zip", folder: {…}, exclude: Array(8), extension: Array(0)}exclude: Array(8)
0: "/home/theo/public_html/mysite/site/assets/cache/WireTempDir"
1: "/home/theo/public_html/mysite/wire"
2: "/home/theo/public_html/mysite/site/assets/backups"
3: "/home/theo/public_html/mysite/site/assets/logs"
4: "/home/theo/public_html/mysite/site/assets/cache"
5: "/home/theo/public_html/mysite/site/assets/sessions"
6: "/home/theo/public_html/mysite/site/assets/files"
7: ""
length: 8__proto__: Array(0)
extension: []length: 0__proto__: Array(0)
filename: "2017-12-08_11-56-07-localhost.pw.zip"
folder: {}__proto__: Object__proto__: Object
log.js:137 $zipinfo:
log.js:137 {files: 4951, dirs: 0, size: 280395773}dirs: 0files: 4951size: 280395773__proto__: Object
log.js:137 $fileinfo:
log.js:137 {zipfile: "/home/theo/public_html/mysite/site/assets/cach…cator-temp/1/2017-12-08_11-56-07-localhost.pw.zip", zipname: "2017-12-08_11-56-07-localhost.pw.zip", structure: {…}}structure: {files: 4951, dirs: 0, size: 280395773}zipfile: "/home/theo/public_html/mysite/site/assets/cache/WireTempDir/.duplicator-temp/1/2017-12-08_11-56-07-localhost.pw.zip"zipname: "2017-12-08_11-56-07-localhost.pw.zip"__proto__: Object

Thank you

  • Thanks 1

Share this post


Link to post
Share on other sites

Strange thing in function zipData

I add these logs around $zip->close()

    ChromePhp::log($zip);
    ChromePhp::log($zip->close());
    ChromePhp::log($zip);
log.js:137 ZipArchive {status: 0, statusSys: 0, numFiles: 4951, filename: "/home/theo/public_html/mysite/site/assets/cach…cator-temp/1/2017-12-08_13-42-33-localhost.pw.zip", comment: ""}
log.js:137 false
log.js:137 ZipArchive {status: 0, statusSys: 0, numFiles: 0, filename: "", comment: ""}

 

$zip->close() returns false.

What could be the reason?

Share this post


Link to post
Share on other sites

OK, some success here: Please add "is_readable" check  in function zipData

 } elseif (is_file($file) && is_readable($file)) {
                            $filecount++;
                            $zip->addFile($file, str_replace($source . '', '', $file));
                            $limitItems++;

Like this it works. If a file is not readable, it silently fails otherwise.

  • Like 3

Share this post


Link to post
Share on other sites

Answering from my mobile.

Good catch ! I will update the module with  your patch, we need now to alert the user that some files are not resdable and thus excluded from the zip. Thanks again @theo!

  • Like 3

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By jaro
      This module (github) does with site/assets/files what Ryan's DatabaseBackups module does with the database:
      Backup site/assets Download ZIP archive Upload ZIP archive Restore site/assets Motivation: This module can be the missing part for projects with content backup responsibility on the client's side: The client will be able to download DB and assets/files snapshots through the backend without filesystem access, thus backing up all content themselves.
      Release state alpha – do not use in production environments.
      Credits for the nice UI go to @ryan – I reused most of it and some other code from the DatabaseBackups module.
    • By David Karich
      ProcessWire InputfieldRepeaterMatrixDuplicate
      Thanks to the great ProModule "RepeaterMatrix" I have the possibility to create complex repeater items. With it I have created a quite powerful page builder. Many different content modules, with many more possible design options. The RepeaterMatrix module supports the cloning of items, but only within the same page. Now I often have the case that very design-intensive pages and items are created. If you want to use a content module on a different page (e.g. in the same design), you have to rebuild each item manually every time.
      This module extends the commercial ProModule "RepeaterMatrix" by the function to duplicate repeater items from one page to another page. The condition is that the target field is the same matrix field from which the item is duplicated. This module is currently understood as proof of concept. There are a few limitations that need to be considered. The intention of the module is that this functionality is integrated into the core of RepeaterMatrix and does not require an extra module.
      Check out the screencast
      What the module can do
      Duplicate a repeater item from one page to another No matter how complex the item is Full support for file and image fields Multilingual support Support of Min and Max settings Live synchronization of clipboard between multiple browser tabs. Copy an item and simply switch the browser tab to the target page and you will immediately see the past button Support of multiple RepeaterMatrix fields on one page Configurable which roles and fields are excluded Duplicated items are automatically pasted to the end of the target field and set to hidden status so that changes are not directly published Automatic clipboard update when other items are picked Automatically removes old clipboard data if it is not pasted within 6 hours Delete clipboard itself by clicking the selected item again Benefit: unbelievably fast workflow and content replication What the module can't do
      Before an item can be duplicated in its current version, the source page must be saved. This means that if you make changes to an item and copy this, the old saved state will be duplicated Dynamic loading is currently not possible. Means no AJAX. When pasting, the target page is saved completely No support for nested repeater items. Currently only first level items can be duplicated. Means a repeater field in a repeater field cannot be duplicated. Workaround: simply duplicate the parent item Dynamic reloading and adding of repeater items cannot be registered. Several interfaces and events from the core are missing. The initialization occurs only once after the page load event Changelog
      1.0.4
      Bug fix: Various bug fixes and improvements in live synchronization Bug fix: Items are no longer inserted when the normal save button is clicked. Only when the past button is explicitly clicked Feature: Support of multiple repeater fields in one page Feature: Support of repeater Min/Max settings Feature: Configurable roles and fields Enhancement: Improved clipboard management Enhancement: Documentation improvement Enhancement: Corrected few typos #1 1.0.3
      Feature: Live synchronization Enhancement: Load the module only in the backend Enhancement: Documentation improvement 1.0.2
      Bug fix: Various bug fixes and improvements in JS functions Enhancement: Documentation improvement Enhancement: Corrected few typos 1.0.1
      Bug fix: Various bug fixes and improvements in the duplication process 1.0.0
      Initial release Support this module
      If this module is useful for you, I am very thankful for your small donation: Donate 5,- Euro (via PayPal – or an amount of your choice. Thank you!)
      Download this module
      > Github: https://github.com/FlipZoomMedia/InputfieldRepeaterMatrixDuplicate
      > PW module directory: – soon –
    • By NorbertH
      Is there a hook to do something right after cloning a page ?
      I want the page to be saved right after cloning it either from the button in the tree or from a lister, because saving the page triggers several calculations that are not triggered by just cloning the page.
       
      Thanks in advance !
    • By John W.
      Question 1
      I recently installed PW 3.0.62 for a new site and also have sites running older version of PW 3.x.
      Can I export the database on an older version of PW 3.x and import it to PW 3.0.62 without any issues?
       
      Question 2
      (This is kind of alternative to the above for long term use - and maybe a better solution...)
      On  the sites I've previously built I have templates (home, basic-page, contact) and fields that I commonly use, such as business_name, phone_1.  The last site I built is running PW 3.0.42.  I was considering cloning this into a local site and running the upgrade module to bring it up to PW 3.0.62. From there on out when I start I new project I could just run the PW upgrade module, copy the folder to the location for the new project and duplicate the database using the new projects name.

      So basically, I'll always keep a "blank slate" site that I can just run the PW upgrade on, then duplicate into a new project. This would cut down on the work and time spent having to re-create these common fields, that I use. From there, I would just add fields, templates, etc, specific for the new website project.

      Is this a sound approach to speed up development?

       
×
×
  • Create New...