Jump to content
flydev

Duplicator: Backup and move sites

Recommended Posts

Thanks @flydev - it's awesome to have that option - not sure about using a "every 1 minute (*/1)" call as the example though :) 

  • Haha 2

Share this post


Link to post
Share on other sites

@flydev - I set up the system cron last night and didn't think too much about, but checking today I see that it didn't work. It turns out the issue is:

$e = new HookEvent();

You just need to add the ProcessWire namespace to the top of the cron.php file and everything works as expected.

  • Like 2

Share this post


Link to post
Share on other sites

Hi @flydev and @kixe - I am using Duplicator and CronjobDatabaseBackup on a site and I would like Duplicator to also send the DB backups created by CronjobDatabaseBackup to Google Drive. The problem is that because by default they both store files in site/assets/backups

My solution has been to move Duplicators backups to /site/assets/duplicator and then uncheck the "Backups" options under the "Files and folders excluded from the package" setting. This should work, but I think perhaps there could be a cleaner solution. Also @flydev - I think the warning about exponential size if unchecking "backups" should have the caveat that it is only relevant if backups are being stored under /sites/assets/backups.

The other quick question - if I set the max number of packages to 10, but leave the "Remove backup packages older than" option to "never", what will happen. I assume that as the 11th is added, the 1st will be removed, but this seems to contradict the "Remove backup packages older than" option. Can you explain exactly what happens here?

Edited by adrian
I'm an idiot :)

Share this post


Link to post
Share on other sites

Sorry, I forgot that Duplicator of course also handles DB backup :)

  • Like 1

Share this post


Link to post
Share on other sites
19 hours ago, adrian said:

I think the warning about exponential size if unchecking "backups" should have the caveat that it is only relevant if backups are being stored under /sites/assets/backups

I will add this check, it make sense to me. Also it might be worth to check if the backup destination is being included - e.g. the destination is in the webroot but not excluded - then if yes, the warning appear, as the package will be saved and the ZIP file will grow.

 

19 hours ago, adrian said:

if I set the max number of packages to 10, but leave the "Remove backup packages older than" option to "never", what will happen.

As you said, the last package will be saved and the oldest package is removed if the limit is reach.

Now I am thinking to make those two settings checking each other giving the priority to the "Remove  backup packages older than" setting.

 

20 hours ago, adrian said:

You just need to add the ProcessWire namespace to the top of the cron.php

I will test it more closely as I would like to keep the compatibility to ProcessWire 2.x in case some people want backup their pw2 website for an eventual upgrade.

 

My brain is overheating today, I'll read again your comments into an hour or so :lol:

  • Like 2
  • Thanks 1

Share this post


Link to post
Share on other sites
1 minute ago, flydev said:

I will test it more closely as I would like to keep the compatibility to ProcessWire 2.x in case some people want backup their pw2 website for an eventual upgrade.

Good point - then simply do this:

$e = class_exists("\ProcessWire\HookEvent") ? new \ProcessWire\HookEvent() : new HookEvent();

 

1 minute ago, flydev said:

My brain is overheating today, I'll read again your comments into an hour or so :lol:

I know that feeling very well - rest up!

  • Like 4

Share this post


Link to post
Share on other sites

Thinking about this a little more - is there any reason to actually instantiate a new HookEvent() ? The cronJob() method doesn't actually make use of the passed $event. Am I missing something?

Share this post


Link to post
Share on other sites

I used it in an earlier version and I left it in case. I did well not to push the new version.

Share this post


Link to post
Share on other sites

Hi

I have a PW 2.7.2. installation.

According to to docs, your module should work with 2.7 resp. 2.7.2 but it does not install because of

const DUP_MIN_VERSION               = '2.7.3';

Is there a technical reason for this, or can I safely change the DUP_MIN_VERSION to '2.7.2' in the module source?

Thank you.

  • Like 1

Share this post


Link to post
Share on other sites

Hi @theo,

you can change it safely, the reason if I remember correctly, is because ProcessWire 2.7.3 was the latest stable version of the 2.x branch - I didn't tested Duplicator on a earlier PW version.

 

Please report here if you could backup it successfully :)

Edited by flydev
precision

Share this post


Link to post
Share on other sites

Thank you.

No, it does not work here right now, but I think it does not have to do with the PW version, because it does not work with 3.0.83 either.

Could it be because I'm using mod_userdir? https://httpd.apache.org/docs/2.4/mod/mod_userdir.html

Where in my case

File root of PW: /home/theo/public_html/pwdev/

Web root of PW: http://localhost/~theo/pwdev/

Thank you.

Share this post


Link to post
Share on other sites

Hi @theo , curious to know what you get when outputing something like

echo "<pre>
path  {$config->paths->templates}
url   {$config->urls->templates}
";

 

  • Like 1

Share this post


Link to post
Share on other sites

Also, could you check the logs please in /your_root_dir/site/assets/logs/duplicator.log to see if there is something revelant ?

Share this post


Link to post
Share on other sites
5 minutes ago, horst said:

Hi @theo , curious to know what you get when outputing something like


echo "<pre>
path  {$config->paths->templates}
url   {$config->urls->templates}
";

 

This:

path  /home/theo/public_html/pwdev/site/templates/
url   /~theo/pwdev/site/templates/

Thank you.

Share this post


Link to post
Share on other sites
6 minutes ago, flydev said:

Also, could you check the logs please in /your_root_dir/site/assets/logs/duplicator.log to see if there is something revelant ?

There is no such file. But there is a warning on the package manager page:

Warning: filesize(): stat failed for /home/theo/public_html/pwdev/site/assets/logs/duplicator.txt in /home/theo/public_html/pwdev/wire/core/FileLog.php on line 225

Warning: filesize(): stat failed for /home/theo/public_html/pwdev/site/assets/logs/duplicator.txt in /home/theo/public_html/pwdev/wire/core/FileLog.php on line 225

Warning: filesize(): stat failed for /home/theo/public_html/pwdev/site/assets/logs/duplicator.txt in /home/theo/public_html/pwdev/wire/core/FileLog.php on line 237

Warning: file(/home/theo/public_html/pwdev/site/assets/logs/duplicator.txt): failed to open stream: Datei oder Verzeichnis nicht gefunden in /home/theo/public_html/pwdev/wire/core/FileLog.php on line 238

And there is a mesage after pressing "Initiate Backup Process"

No route, document, custom route or redirect is matching the request: /~theo/pwdev/~theopwdevprocesswire/setup/duplicator/?action=none | Specific ERROR: No route matched the request

 

 

Share this post


Link to post
Share on other sites

Thanks - ok, there is no file because Duplicator has not run at least one time. Maybe the PHP script is killed abruptly. Please check your PM.

 

edit:

I didn't saw the last error message, there is something broken with the path.

Share this post


Link to post
Share on other sites

Yes, and I can tell you that it works, when I configure a virtual host in Apache and add "127.0.0.1    pwdev.ch" in the hosts file.

<VirtualHost *:80>
    ServerAdmin some@some.ch
    ServerAlias pwdev.ch
    ServerName www.pwdev.ch
    DocumentRoot /home/theo/public_html/pwdev
</VirtualHost>

Then running under http://pwdev.ch/processwire/ everything is fine. 

So it has to do with mod_userdir.

Thank you.

  • Like 1

Share this post


Link to post
Share on other sites

Thanks to you for reporting it back. I will test it on a server with mod_userdir installed today.

  • Like 1

Share this post


Link to post
Share on other sites

Thanks @flydev for your work on this module. I keep getting the following error when trying to use FTP:

"[FTP] error: invalid file 2017-12-04_07-01-36-glosas.anle.us.package.zip, bad extension"

I don't understand what "bad extension" means. I've tried leaving the Name field for the backup file blank (default state), and changing it also (which by the way seems to have no effect), but I get the same error.

Another question: are you going to upload the Google Drive SDK? I get a 404 error when clicking on "you can download it here..."

Thanks.

  • Like 1

Share this post


Link to post
Share on other sites
14 minutes ago, jacmaes said:

I don't understand what "bad extension" means.

Never saw this issue with a FTP server, maybe its a specific settings of the server ?

Do you have good results uploading a conventional file, like a "foobar.txt", an image or even a  zip file ?

I think I found the issue, it has nothing to do with server, I am dumb. I will push a fix.

 

14 minutes ago, jacmaes said:

are you going to upload the Google Drive SDK?

If I have the time, i will upload the SDKs tonight - I have to update and test them before.

 

---

 

56 minutes ago, theo said:

So it has to do with mod_userdir.

I can confirm this issue as I could reproduce it right now, I will find a fix.

  • Like 1

Share this post


Link to post
Share on other sites

@theo :  I just pushed a fix - now Duplicator work with Apache mod_userdir.

If you can upgrade the module to the latest version from Github or the module directory (version 1.1.4) and report back, I will be happy ;)

 

 

edit:  don't forget to change the const DUP_MIN_VERSION

  • Like 1

Share this post


Link to post
Share on other sites
8 minutes ago, flydev said:

@theo :  I just pushed a fix - now Duplicator work with Apache mod_userdir.

If you can upgrade the module to the latest version from Github or the module directory and report back, I will be happy ;)

 

Thank you.

It works now with mod_userdir and PW 3.0.83, but it does still not work with 2.7.2

The log says:

2017-12-07 15:23:13:  Package build failed.
2017-12-07 15:23:13:  An error occured during package build.
2017-12-07 15:23:13:  An error occured while building the ProcessWire structure.

 

Share this post


Link to post
Share on other sites

I just tested Duplicator on 2.7.2 fresh install, everything is fine.

Testing with mod_userdir... I will edit this post.

 

Edit: @theo  yep it work on my side with mod_userdir, ProcessWire 2.7.2, Duplicator 1.1.4

Maybe removing the cache and/or a modules refresh could fix it.

Share this post


Link to post
Share on other sites
20 minutes ago, flydev said:

I just tested Duplicator on 2.7.2 fresh install, everything is fine.

Testing with mod_userdir... I will edit this post.

 

Edit: @theo  yep it work on my side with mod_userdir, ProcessWire 2.7.2, Duplicator 1.1.4

Maybe removing the cache and/or a modules refresh could fix it.

Thank you, but no, same problem.

Maybe there is something "wrong" with my installation.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By David Karich
      ProcessWire InputfieldRepeaterMatrixDuplicate
      Thanks to the great ProModule "RepeaterMatrix" I have the possibility to create complex repeater items. With it I have created a quite powerful page builder. Many different content modules, with many more possible design options. The RepeaterMatrix module supports the cloning of items, but only within the same page. Now I often have the case that very design-intensive pages and items are created. If you want to use a content module on a different page (e.g. in the same design), you have to rebuild each item manually every time.
      This module extends the commercial ProModule "RepeaterMatrix" by the function to duplicate repeater items from one page to another page. The condition is that the target field is the same matrix field from which the item is duplicated. This module is currently understood as proof of concept. There are a few limitations that need to be considered. The intention of the module is that this functionality is integrated into the core of RepeaterMatrix and does not require an extra module.
      Check out the screencast
      What the module can do
      Duplicate a repeater item from one page to another No matter how complex the item is Full support for file and image fields Multilingual support Support of Min and Max settings Live synchronization of clipboard between multiple browser tabs. Copy an item and simply switch the browser tab to the target page and you will immediately see the past button Support of multiple RepeaterMatrix fields on one page Configurable which roles and fields are excluded Duplicated items are automatically pasted to the end of the target field and set to hidden status so that changes are not directly published Automatic clipboard update when other items are picked Automatically removes old clipboard data if it is not pasted within 6 hours Delete clipboard itself by clicking the selected item again Benefit: unbelievably fast workflow and content replication What the module can't do
      Before an item can be duplicated in its current version, the source page must be saved. This means that if you make changes to an item and copy this, the old saved state will be duplicated Dynamic loading is currently not possible. Means no AJAX. When pasting, the target page is saved completely No support for nested repeater items. Currently only first level items can be duplicated. Means a repeater field in a repeater field cannot be duplicated. Workaround: simply duplicate the parent item Dynamic reloading and adding of repeater items cannot be registered. Several interfaces and events from the core are missing. The initialization occurs only once after the page load event Changelog
      1.0.4
      Bug fix: Various bug fixes and improvements in live synchronization Bug fix: Items are no longer inserted when the normal save button is clicked. Only when the past button is explicitly clicked Feature: Support of multiple repeater fields in one page Feature: Support of repeater Min/Max settings Feature: Configurable roles and fields Enhancement: Improved clipboard management Enhancement: Documentation improvement Enhancement: Corrected few typos #1 1.0.3
      Feature: Live synchronization Enhancement: Load the module only in the backend Enhancement: Documentation improvement 1.0.2
      Bug fix: Various bug fixes and improvements in JS functions Enhancement: Documentation improvement Enhancement: Corrected few typos 1.0.1
      Bug fix: Various bug fixes and improvements in the duplication process 1.0.0
      Initial release Support this module
      If this module is useful for you, I am very thankful for your small donation: Donate 5,- Euro (via PayPal – or an amount of your choice. Thank you!)
      Download this module
      > Github: https://github.com/FlipZoomMedia/InputfieldRepeaterMatrixDuplicate
      > PW module directory: https://modules.processwire.com/modules/inputfield-repeater-matrix-duplicate/
    • By jaro
      This module (github) does with site/assets/files what Ryan's DatabaseBackups module does with the database:
      Backup site/assets Download ZIP archive Upload ZIP archive Restore site/assets Motivation: This module can be the missing part for projects with content backup responsibility on the client's side: The client will be able to download DB and assets/files snapshots through the backend without filesystem access, thus backing up all content themselves.
      Release state alpha – do not use in production environments.
      Credits for the nice UI go to @ryan – I reused most of it and some other code from the DatabaseBackups module.
    • By NorbertH
      Is there a hook to do something right after cloning a page ?
      I want the page to be saved right after cloning it either from the button in the tree or from a lister, because saving the page triggers several calculations that are not triggered by just cloning the page.
       
      Thanks in advance !
    • By John W.
      Question 1
      I recently installed PW 3.0.62 for a new site and also have sites running older version of PW 3.x.
      Can I export the database on an older version of PW 3.x and import it to PW 3.0.62 without any issues?
       
      Question 2
      (This is kind of alternative to the above for long term use - and maybe a better solution...)
      On  the sites I've previously built I have templates (home, basic-page, contact) and fields that I commonly use, such as business_name, phone_1.  The last site I built is running PW 3.0.42.  I was considering cloning this into a local site and running the upgrade module to bring it up to PW 3.0.62. From there on out when I start I new project I could just run the PW upgrade module, copy the folder to the location for the new project and duplicate the database using the new projects name.

      So basically, I'll always keep a "blank slate" site that I can just run the PW upgrade on, then duplicate into a new project. This would cut down on the work and time spent having to re-create these common fields, that I use. From there, I would just add fields, templates, etc, specific for the new website project.

      Is this a sound approach to speed up development?

       
×
×
  • Create New...