☁️ Duplicator: Backup and move sites

[Peter Knight]

Yes it’s only for year 1. But a good deal if you’re intending on a paid plan anyway.  ?

[Karl_T]

I have encountered an issue for Duplicator ver 1.2.9.

I am moving a site from localhost(windows 10) to Amazon Linux. I use RDS, which is a standalone server for database, so the dbHost is not localhost.

The issue happened in the installation process after entering the database data and proceed. There's an error telling config file cannot be found. Then I go to config.php, there are at least 4 places are using old information. They are dbHost, dbName, dbUser, dbPass and httpHosts. I think for some reasons the config.php did not update at all after the installation complete. After I manually update the fields, the site works fine now. This means database import is working fine. I wonder if it is the dbHost matters. It is a long URL ended with xxx.rds.amazonaws.com.

[flydev]

Hi @Karl_T sorry for that and thanks for reporting this issue, I think I have an idea why is happening - checking this afternoon.

[theo]

Hi

Latest Duplicator does not work here.

It does not include ./wire and other stuff.

Version 1.1.7 works.

Thank you.

[szabesz]

I am running version 1.2.9 without issues on two sites.

13 minutes ago, theo said:

It does not include ./wire and other stuff.

You might want to double check and re-save your settings. There might have been changes regarding the way the module stores what to include and what not to.

[pwired]

If duplicator fails there is no sweat at all. Simply upload your site folder and database.
I never got it why duplicator has an advantage over manually uploading your site folder and database.
The manual down and upload is compatible with all versions of processwire
I also don't see any time saving when using it.

[szabesz]

43 minutes ago, pwired said:

I never got it why duplicator has an advantage over manually uploading your site folder and database.

It all depends on your workflow. Also, Duplicator is not just about cloning a site but for creating backups. Normally I use it for daily backups and/or for quick backups before "performing sensitive changes", just to be on the safe side...

[pwired]

Quote

It all depends on your workflow. Also, Duplicator is not just about cloning a site but for creating backups. Normally I use it for daily backups and/or for quick backups before "performing sensitive changes", just to be on the safe side...

Maybe I still haven't enough websites on my shoulders to see the use of it, so I should install and evaluate it. What about scripting your cpanel over the web ? Does anyone know if there exists a scripting language for automating cpanel tasks ? I don't mean cronjobs but something like cUrl.

[szabesz]

7 minutes ago, pwired said:

What about scripting your cpanel over the web ? Does anyone know if there exists a scripting language for automating cpanel tasks ? I don't mean cronjobs but something like cUrl.

I always insist on having SSH no matter what because BASH is also in my toolset. I either clone a complete site with my own script or just the database. I make admin changes in production (if I'm not sure about the results in advance, then I perform tests in dev first) and after that I use my cloning script to replace the dev database from prod which takes only a few (4-7) seconds to do if one does not have to deal with a huge dbs (which I currently don't). 

https://processwire.com/talk/topic/14837-best-practise-to-implement-changes-in-pw-from-development-to-production/?do=findComment&comment=133726

https://processwire.com/talk/topic/14837-best-practise-to-implement-changes-in-pw-from-development-to-production/?do=findComment&comment=133741

 

 

[pwired]

Quote

   SSH . . .

That makes me thinking about writing your own scripts that drive cPanel SSH Commands.
Maybe with AutoHotKey for windows. That would be a cool way to automate and task schedule
backups, down and uploads.

[theo]

3 hours ago, szabesz said:

I am running version 1.2.9 without issues on two sites.

You might want to double check and re-save your settings. There might have been changes regarding the way the module stores what to include and what not to.

I have completely uninstalled and deleted it, even checked in the db for the module entry, then downloaded and installed the latest version (1.2.9) and no, it does not work as before.

The wire folder is not included etc.

The package is only 2.11 MB instead of more than 20 MB before.

Thank you.

[theo]

@szabesz Probably you remember my "special" setup.

I had some problems before 1.1.6 in the area of mod_userdir and this:

 

 

Maybe the current problem is related?

[theo]

OK, I had a quick look at the source of Duplicator.module

On line 640, there is a "isWritable" check:

if(!$f->isReadable() || !$f->isWritable()) { continue; }

Why does the file to copy have to be writable?

My "wire" folder is not writable by "wwwrun" because it is not necessary, it has to be readable though.

If I remove this check, it works as before.

Thank you.

[theo]

There is another issue in my case:

The parent folder is included in v 1.2.9.

Probably has to do with mod_userdir.

So it does not work because it is extracting everything to /parentfolder/wire etc.

As the above issue, this was all working in v 1.1.7

[adrian]

Not sure if it's all servers, but I am finding the downloading a package from Setup > Duplicator is resulting in partial downloads. I don't have time to investigate at the moment, but just a heads up in case someone is trusting the download to be complete.

[szabesz]

14 minutes ago, adrian said:

Setup > Duplicator is resulting in partial downloads. I don't have time to investigate at the moment, but just a heads up in case someone is trusting the download to be complete.

I guess it depends on the file size of the ZIP file. Big files cannot be downloaded for example if not enough memory is allocated for the process. I always download the ZIP files via FTP which is reliable in my case.

[flydev]

On 5/2/2018 at 10:48 AM, theo said:

Why does the file to copy have to be writable?

My "wire" folder is not writable by "wwwrun" because it is not necessary, it has to be readable though. 

@theo The new function is based on the official WireFileTools::zip() function and I didn't saw this check. Nice catch ?

 

On 5/2/2018 at 5:19 PM, theo said:

There is another issue in my case:

The parent folder is included in v 1.2.9.

Probably has to do with mod_userdir.

It has not do with the issue I remember. I just checked and its a silly mistake I made while updating the module. Thanks again @theo . It will be fixed in Duplicator-1.2.10

 

On 5/2/2018 at 9:32 PM, adrian said:

Not sure if it's all servers, but I am finding the downloading a package from Setup > Duplicator is resulting in partial downloads. I don't have time to investigate at the moment, but just a heads up in case someone is trusting the download to be complete.

I never ran into into this issue but it look like it can happen. I have a site with 600MB stuff and his database size at 1GB+, I have no timeout building the package or downloading issue. To me, it look like a server settings.

If you can, please send me the server details so I can try to reproduce it.

PS: still didn't found the time to check what is going on GoogleDrive.

 

@szabesz any feedback about Dropbox v2 ? ?

 

[szabesz]

1 hour ago, flydev said:

any feedback about Dropbox v2 ?

I do not have too much extra time these days so I postponed testing it, also because you mentioned possible idrive support and I thought I would wait for that too, I was thinking about testing both "parallel".

[flydev]

A small news about Duplicator and why its not updated recently:

On the current state of the module, it is usable for small website, I mean small database (I am aware of the issue which include the  parent directory in the compressed package which make the package non deployable by the installer).

As stated by Ryan itself, If you are working with really large files, it may be better to perform that task directly in PhpMyAdmin or the native MySQL client.

And as I am working now with giga-sized databases, its the time to rewrite this module. So what is planned ? 

 

Duplicator will be made on top of PHPBU in the hope to provide to the community a professional focused backup service for your websites.

Quote

PHPBU is a php framework that creates and encrypts backups and moves your backups to other servers or cloud services
and assists you monitor your backup creation

More information :

• Future requirement: PHP 7.x.x
• The official repo : https://github.com/sebastianfeldmann/phpbu

 

Any suggestion welcome. Have a nice day ! ✌️

[szabesz]

On 6/21/2018 at 11:04 AM, flydev said:

Any suggestion welcome. Have a nice day ! ✌️

This sounds awesome! I can offer my humble testing skills even though I'm not having too much free time these days but this project is more important than ever (thinking of GDPR compliant backups, ie. encryption for backups).

Last time Flydev could not reach out to too many active testers, so now I encourage anyone reading these lines to join in and offer testing – maybe even development – help to Flydev in advance if you can really do it when he needs it.

Also, please do not just hit the like button but reply with suggestions too ? BTW, my first suggestions is: besides being GUI friendly, the new module could also be cli friendly: meaning wireshell integration. 

[arjen]

I'm in ?

[horst]

On 6/25/2018 at 3:15 PM, arjen said:

I'm in ?

I'm too.

[adrian]

I'm three!

[dotnetic]

Count me in as a tester.

[schwarzdesign]

Quick question, the automatic backups through LazyCron aren't working for me. Manual backups are working fine (both locally and with remote FTP backup), but the automatic backups won't trigger at all.

Here are my settings:

• Backup mode: LazyCron - Triggered by a pageview (installed)
• Event trigger: 1 hour
• Remove backups older than: 1 year

I have waited for several hours, the site had traffic during that time, and I also visited some pages as a logged-in administrator and as an anonymous visitor. But no backups occured, there are also no log entries at all, so it doesn't look like it just bugged out. Do I need to configure anything else? LazyCron is installed, it doesn't have any module options though. Or do I need to manually call the LazyCron hooks in ready.php or something like that? Not sure what I am missing.

Thanks!