Jump to content
flydev 👊🏻

☁️ Duplicator: Backup and move sites

Recommended Posts

Thanks @flydev - it's awesome to have that option - not sure about using a "every 1 minute (*/1)" call as the example though :) 

  • Haha 2

Share this post


Link to post
Share on other sites

@flydev - I set up the system cron last night and didn't think too much about, but checking today I see that it didn't work. It turns out the issue is:

$e = new HookEvent();

You just need to add the ProcessWire namespace to the top of the cron.php file and everything works as expected.

  • Like 2

Share this post


Link to post
Share on other sites

Hi @flydev and @kixe - I am using Duplicator and CronjobDatabaseBackup on a site and I would like Duplicator to also send the DB backups created by CronjobDatabaseBackup to Google Drive. The problem is that because by default they both store files in site/assets/backups

My solution has been to move Duplicators backups to /site/assets/duplicator and then uncheck the "Backups" options under the "Files and folders excluded from the package" setting. This should work, but I think perhaps there could be a cleaner solution. Also @flydev - I think the warning about exponential size if unchecking "backups" should have the caveat that it is only relevant if backups are being stored under /sites/assets/backups.

The other quick question - if I set the max number of packages to 10, but leave the "Remove backup packages older than" option to "never", what will happen. I assume that as the 11th is added, the 1st will be removed, but this seems to contradict the "Remove backup packages older than" option. Can you explain exactly what happens here?

Edited by adrian
I'm an idiot :)

Share this post


Link to post
Share on other sites

Sorry, I forgot that Duplicator of course also handles DB backup :)

  • Like 1

Share this post


Link to post
Share on other sites
19 hours ago, adrian said:

I think the warning about exponential size if unchecking "backups" should have the caveat that it is only relevant if backups are being stored under /sites/assets/backups

I will add this check, it make sense to me. Also it might be worth to check if the backup destination is being included - e.g. the destination is in the webroot but not excluded - then if yes, the warning appear, as the package will be saved and the ZIP file will grow.

 

19 hours ago, adrian said:

if I set the max number of packages to 10, but leave the "Remove backup packages older than" option to "never", what will happen.

As you said, the last package will be saved and the oldest package is removed if the limit is reach.

Now I am thinking to make those two settings checking each other giving the priority to the "Remove  backup packages older than" setting.

 

20 hours ago, adrian said:

You just need to add the ProcessWire namespace to the top of the cron.php

I will test it more closely as I would like to keep the compatibility to ProcessWire 2.x in case some people want backup their pw2 website for an eventual upgrade.

 

My brain is overheating today, I'll read again your comments into an hour or so :lol:

  • Like 2
  • Thanks 1

Share this post


Link to post
Share on other sites
1 minute ago, flydev said:

I will test it more closely as I would like to keep the compatibility to ProcessWire 2.x in case some people want backup their pw2 website for an eventual upgrade.

Good point - then simply do this:

$e = class_exists("\ProcessWire\HookEvent") ? new \ProcessWire\HookEvent() : new HookEvent();

 

1 minute ago, flydev said:

My brain is overheating today, I'll read again your comments into an hour or so :lol:

I know that feeling very well - rest up!

  • Like 4

Share this post


Link to post
Share on other sites

Thinking about this a little more - is there any reason to actually instantiate a new HookEvent() ? The cronJob() method doesn't actually make use of the passed $event. Am I missing something?

Share this post


Link to post
Share on other sites

Hi

I have a PW 2.7.2. installation.

According to to docs, your module should work with 2.7 resp. 2.7.2 but it does not install because of

const DUP_MIN_VERSION               = '2.7.3';

Is there a technical reason for this, or can I safely change the DUP_MIN_VERSION to '2.7.2' in the module source?

Thank you.

  • Like 1

Share this post


Link to post
Share on other sites

Hi @theo,

you can change it safely, the reason if I remember correctly, is because ProcessWire 2.7.3 was the latest stable version of the 2.x branch - I didn't tested Duplicator on a earlier PW version.

 

Please report here if you could backup it successfully :)

Edited by flydev
precision

Share this post


Link to post
Share on other sites

Thank you.

No, it does not work here right now, but I think it does not have to do with the PW version, because it does not work with 3.0.83 either.

Could it be because I'm using mod_userdir? https://httpd.apache.org/docs/2.4/mod/mod_userdir.html

Where in my case

File root of PW: /home/theo/public_html/pwdev/

Web root of PW: http://localhost/~theo/pwdev/

Thank you.

Share this post


Link to post
Share on other sites

Hi @theo , curious to know what you get when outputing something like

echo "<pre>
path  {$config->paths->templates}
url   {$config->urls->templates}
";

 

  • Like 1

Share this post


Link to post
Share on other sites

Also, could you check the logs please in /your_root_dir/site/assets/logs/duplicator.log to see if there is something revelant ?

Share this post


Link to post
Share on other sites
5 minutes ago, horst said:

Hi @theo , curious to know what you get when outputing something like


echo "<pre>
path  {$config->paths->templates}
url   {$config->urls->templates}
";

 

This:

path  /home/theo/public_html/pwdev/site/templates/
url   /~theo/pwdev/site/templates/

Thank you.

Share this post


Link to post
Share on other sites
6 minutes ago, flydev said:

Also, could you check the logs please in /your_root_dir/site/assets/logs/duplicator.log to see if there is something revelant ?

There is no such file. But there is a warning on the package manager page:

Warning: filesize(): stat failed for /home/theo/public_html/pwdev/site/assets/logs/duplicator.txt in /home/theo/public_html/pwdev/wire/core/FileLog.php on line 225

Warning: filesize(): stat failed for /home/theo/public_html/pwdev/site/assets/logs/duplicator.txt in /home/theo/public_html/pwdev/wire/core/FileLog.php on line 225

Warning: filesize(): stat failed for /home/theo/public_html/pwdev/site/assets/logs/duplicator.txt in /home/theo/public_html/pwdev/wire/core/FileLog.php on line 237

Warning: file(/home/theo/public_html/pwdev/site/assets/logs/duplicator.txt): failed to open stream: Datei oder Verzeichnis nicht gefunden in /home/theo/public_html/pwdev/wire/core/FileLog.php on line 238

And there is a mesage after pressing "Initiate Backup Process"

No route, document, custom route or redirect is matching the request: /~theo/pwdev/~theopwdevprocesswire/setup/duplicator/?action=none | Specific ERROR: No route matched the request

 

 

Share this post


Link to post
Share on other sites

Thanks - ok, there is no file because Duplicator has not run at least one time. Maybe the PHP script is killed abruptly. Please check your PM.

 

edit:

I didn't saw the last error message, there is something broken with the path.

Share this post


Link to post
Share on other sites

Yes, and I can tell you that it works, when I configure a virtual host in Apache and add "127.0.0.1    pwdev.ch" in the hosts file.

<VirtualHost *:80>
    ServerAdmin some@some.ch
    ServerAlias pwdev.ch
    ServerName www.pwdev.ch
    DocumentRoot /home/theo/public_html/pwdev
</VirtualHost>

Then running under http://pwdev.ch/processwire/ everything is fine. 

So it has to do with mod_userdir.

Thank you.

  • Like 1

Share this post


Link to post
Share on other sites

Thanks @flydev for your work on this module. I keep getting the following error when trying to use FTP:

"[FTP] error: invalid file 2017-12-04_07-01-36-glosas.anle.us.package.zip, bad extension"

I don't understand what "bad extension" means. I've tried leaving the Name field for the backup file blank (default state), and changing it also (which by the way seems to have no effect), but I get the same error.

Another question: are you going to upload the Google Drive SDK? I get a 404 error when clicking on "you can download it here..."

Thanks.

  • Like 1

Share this post


Link to post
Share on other sites
14 minutes ago, jacmaes said:

I don't understand what "bad extension" means.

Never saw this issue with a FTP server, maybe its a specific settings of the server ?

Do you have good results uploading a conventional file, like a "foobar.txt", an image or even a  zip file ?

I think I found the issue, it has nothing to do with server, I am dumb. I will push a fix.

 

14 minutes ago, jacmaes said:

are you going to upload the Google Drive SDK?

If I have the time, i will upload the SDKs tonight - I have to update and test them before.

 

---

 

56 minutes ago, theo said:

So it has to do with mod_userdir.

I can confirm this issue as I could reproduce it right now, I will find a fix.

  • Like 1

Share this post


Link to post
Share on other sites

@theo :  I just pushed a fix - now Duplicator work with Apache mod_userdir.

If you can upgrade the module to the latest version from Github or the module directory (version 1.1.4) and report back, I will be happy ;)

 

 

edit:  don't forget to change the const DUP_MIN_VERSION

  • Like 1

Share this post


Link to post
Share on other sites
8 minutes ago, flydev said:

@theo :  I just pushed a fix - now Duplicator work with Apache mod_userdir.

If you can upgrade the module to the latest version from Github or the module directory and report back, I will be happy ;)

 

Thank you.

It works now with mod_userdir and PW 3.0.83, but it does still not work with 2.7.2

The log says:

2017-12-07 15:23:13:  Package build failed.
2017-12-07 15:23:13:  An error occured during package build.
2017-12-07 15:23:13:  An error occured while building the ProcessWire structure.

 

Share this post


Link to post
Share on other sites

I just tested Duplicator on 2.7.2 fresh install, everything is fine.

Testing with mod_userdir... I will edit this post.

 

Edit: @theo  yep it work on my side with mod_userdir, ProcessWire 2.7.2, Duplicator 1.1.4

Maybe removing the cache and/or a modules refresh could fix it.

Share this post


Link to post
Share on other sites
20 minutes ago, flydev said:

I just tested Duplicator on 2.7.2 fresh install, everything is fine.

Testing with mod_userdir... I will edit this post.

 

Edit: @theo  yep it work on my side with mod_userdir, ProcessWire 2.7.2, Duplicator 1.1.4

Maybe removing the cache and/or a modules refresh could fix it.

Thank you, but no, same problem.

Maybe there is something "wrong" with my installation.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By David Karich
      ProcessWire InputfieldRepeaterMatrixDuplicate
      Thanks to the great ProModule "RepeaterMatrix" I have the possibility to create complex repeater items. With it I have created a quite powerful page builder. Many different content modules, with many more possible design options. The RepeaterMatrix module supports the cloning of items, but only within the same page. Now I often have the case that very design-intensive pages and items are created. If you want to use a content module on a different page (e.g. in the same design), you have to rebuild each item manually every time.
      This module extends the commercial ProModule "RepeaterMatrix" by the function to duplicate repeater items from one page to another page. The condition is that the target field is the same matrix field from which the item is duplicated. This module is currently understood as proof of concept. There are a few limitations that need to be considered. The intention of the module is that this functionality is integrated into the core of RepeaterMatrix and does not require an extra module.
      Check out the screencast
      What the module can do
      Duplicate multible repeater items from one page to another No matter how complex the item is Full support for file and image fields Multilingual support Support of Min and Max settings Live synchronization of clipboard between multiple browser tabs. Copy an item and simply switch the browser tab to the target page and you will immediately see the past button Support of multiple RepeaterMatrix fields on one page Configurable which roles and fields are excluded Configurable dialogs for copy and paste Duplicated items are automatically pasted to the end of the target field and set to hidden status so that changes are not directly published Automatic clipboard update when other items are picked Automatically removes old clipboard data if it is not pasted within 6 hours Delete clipboard itself by clicking the selected item again Benefit: unbelievably fast workflow and content replication What the module can't do
      Before an item can be duplicated in its current version, the source page must be saved. This means that if you make changes to an item and copy this, the old saved state will be duplicated Dynamic loading is currently not possible. Means no AJAX. When pasting, the target page is saved completely No support for nested repeater items. Currently only first level items can be duplicated. Means a repeater field in a repeater field cannot be duplicated. Workaround: simply duplicate the parent item Dynamic reloading and adding of repeater items cannot be registered. Several interfaces and events from the core are missing. The initialization occurs only once after the page load event Attention, please note!
      Nested repeaters cannot be supported technically. Therefore a check is made to prevent this. However, a nested repeater can only be detected if the field name ends for example with "_repeater1234". For example, if your MatrixRepeater field is named like this: "content_repeater" or "content_repeater123", this field is identified as nested and the module does not load. In version 2.0.1 the identification has been changed so that a field ending with the name repeater is only detected as nested if at least a two-digit number sequence follows. But to avoid this problem completely, make sure that your repeater matrix field does NOT end with the name "repeater".
      Changelog
       
      2.0.1
      Bug fix: Thanks to @ngrmm I could discover a bug which causes that the module cannot be loaded if the MatrixRepeater field ends with the name "repeater". The code was adjusted and information about the problem was provided 2.0.0
      Feature: Copy multiple items at once! The fundament for copying multiple items was created by @Autofahrn - THX! Feature: Optionally you can disable the copy and/or paste dialog Bug fix: A fix suggestion when additional and normal repeater fields are present was contributed by @joshua - THX! 1.0.4
      Bug fix: Various bug fixes and improvements in live synchronization Bug fix: Items are no longer inserted when the normal save button is clicked. Only when the past button is explicitly clicked Feature: Support of multiple repeater fields in one page Feature: Support of repeater Min/Max settings Feature: Configurable roles and fields Enhancement: Improved clipboard management Enhancement: Documentation improvement Enhancement: Corrected few typos #1 1.0.3
      Feature: Live synchronization Enhancement: Load the module only in the backend Enhancement: Documentation improvement 1.0.2
      Bug fix: Various bug fixes and improvements in JS functions Enhancement: Documentation improvement Enhancement: Corrected few typos 1.0.1
      Bug fix: Various bug fixes and improvements in the duplication process 1.0.0
      Initial release Support this module
      If this module is useful for you, I am very thankful for your small donation: Donate 5,- Euro (via PayPal – or an amount of your choice. Thank you!)
      Download this module (Version 2.0.1)
      > Github: https://github.com/FlipZoomMedia/InputfieldRepeaterMatrixDuplicate
      > PW module directory: https://modules.processwire.com/modules/inputfield-repeater-matrix-duplicate/
      > Old stable version (1.0.4): https://github.com/FlipZoomMedia/InputfieldRepeaterMatrixDuplicate/releases/tag/1.0.4
    • By jaro
      This module (github) does with site/assets/files what Ryan's DatabaseBackups module does with the database:
      Backup site/assets Download ZIP archive Upload ZIP archive Restore site/assets Motivation: This module can be the missing part for projects with content backup responsibility on the client's side: The client will be able to download DB and assets/files snapshots through the backend without filesystem access, thus backing up all content themselves.
      Release state alpha – do not use in production environments.
      Credits for the nice UI go to @ryan – I reused most of it and some other code from the DatabaseBackups module.
    • By NorbertH
      Is there a hook to do something right after cloning a page ?
      I want the page to be saved right after cloning it either from the button in the tree or from a lister, because saving the page triggers several calculations that are not triggered by just cloning the page.
       
      Thanks in advance !
×
×
  • Create New...