☁️ Duplicator: Backup and move sites

[flydev]

Now, the "package manager";

From this window, you can create or delete a package, and also download a packages from the "server backups folder" to your computer. But the best feature is that you can "sync" packages stored on a third-party provider. This mean that the module will check for existing packages on GoogleDrive, Dropbox or whatever (it depend on how you configured Duplicator) and list them so you can delete or download them to your computer, and deploy your nice website in a minute!

In the following example, I have packages on my Local Server, on GoogleDrive and Dropbox :

 

[flydev]

As I am writing the documentation, if you guys have already some questions before I release the module, I will be happy to hear and include them in the FAQ

 

[Ivan Gretsky]

Hey, @flydev! I am sure those that are not beta testers will have most questions, so better ask us, eagerly waiting for the release, after it happens)
Glad you are back here, by the way!

[flydev]

@everyone : More beta-tester needed !

 

If you have a bit time, a local or staging web server - feel free to test it on a prod server - and you want to try this module, please send me a private message ! This module can't harm your server

It can be tested on the following ProcessWire version:

• 2.7.2 (no Composer here)
• 2.8.x
• 3.0.x

 

@Ivan Gretsky wanna test it ?

 

[flydev]

I feel sorry again to make you guys waiting again, I got some works to finish asap, but theses tasks made me test Duplicator extensively and I think its ready for the release. It will not work for each hosting providers, but its a start.

Finally, it will come with a new feature: AutoDeploy.

With this feature, you will configure the FTP account and database informations, and the module will deploy automatically the web-site without having to run the installer manually.

[Zeka]

Hi @flydev

Will it be free/public or commercial module? 

[flydev]

Hey Zeka, the module will be free for the community.

[horst]

@flydev 

Do your need one more beta tester? ?

[antpre]

Hi Flydev,

Thanks again, it looks very promising.

One question, would there be a way to have some sort of package profiles. I mean for example, may be I want to have a package profile that backups every day on the server excluding some large media files and an other package profile that backups everyweek with all the file on a S3 bucket.

Does that make sense (for me it does ;) ?

Regards

[flydev]

1 hour ago, antpre said:

Does that make sense (for me it does ;) ?

Yes it make sense, and if I remember correctly we already discussed through PM about this feature. Its not implemented but on the todo list now.

[flydev]

1 hour ago, horst said:

@flydev 

Do your need one more beta tester? ?

I'll give you the url tonight

[Zeka]

26 minutes ago, flydev said:

I'll give you the url tonight

@flydev

I also would like to participate in testing if needed

[antpre]

Thanks Flydev.

That's a good news.

[dragan]

@flydev

I'd like to test too. 

(Especially after I tried to set up backup/sync cron jobs via a server's crappy control panel where everything just works half-baked...)

[jeve]

@flydev : More beta-tester needed !

Hello
can I test your module? I have a lot of sites , someone heavy, so I can test and report

[flydev]

 

There you can download the module : https://github.com/flydev-fr/Duplicator

 

The documentation is incomplete, you have for now to explore the settings on your own. The install instructions are written but the SDKs are not uploaded yet, which mean that AmazonS3, Dropbox and GoogleDrive are only available by installing them through composer. I will upload them tomorrow but they are not recently tested.

Anyway, there is no requirements to run Duplicator as is. Just install Duplicator and Duplicator - Packages Manager then go to Setup > Duplicator and build your package by clicking the button. If everything run smooth, you will see the package listed and you will be able to download it directly and also the installer.

To restore a website, just upload the package and the installer then go to yourwebsite.com/installer.php then follow the instructions.

PLEASE, do not forget - after the end of the process - to delete the installer.php file and your package (this will be fixed in the v0.0.45).

 

It should work with default theme and the UIkit theme.

A dedicated thread will be created in the modules forum soon. Do not hesitate to share here your results by providing your hosting type / provider / config, etc.

Edited November 29, 2017 by flydev
logo (made by @szabesz)

[szabesz]

@flydev Thank you so much for this module! Just a side-note: please consider turning the next version number to at least 0.1.0  or better yet 1.0.0 (I do not think this module is in beta state as it is stable. It might lack some functions you have not yet found the time to finish but those can be pointed out in the read me and not in the version number  ) We have a recent discussion about this and it seems to be beneficial to use semver 2.0 : https://processwire.com/talk/topic/17767-module-release-notes/?do=findComment&comment=156350

 

And for those who doesn't already know I would like to point out that Duplicator is not just about duplicating but making regular backups (via (lazy)cron).

Edited November 29, 2017 by szabesz
typo

[bernhard]

hi flydev

thanks for releasing this

i just tried it and i got multiple errors "call to undefined function _(...". i changed all occurences from _( to __( and it worked. what is this single underscore function that you are using for translations? same for the process module.

also i get an "unrecognized path" for http://duplicator.dev/processwire/setup/Duplicator/?action=packages

can you set the process module to be installed automatically by the main module please?  

the backup worked like a charm. the installer seems to be quite similar to my kickstart project. i think there is great potential to combine both tools!

unfortunately i got an error on the last step:

the problem seems to be the config.php

/**
 * Installer: Database Configuration
 * 
 */
$config->dbHost = 'localhost';
$config->dbName = 'localhost';
$config->dbUser = 'localhost';
$config->dbPass = 'localhost';
$config->dbPort = 'localhost';

/**
 * Installer: User Authentication Salt 
 * 
 * Must be retained if you migrate your site from one server to another
 * 
 */
$config->userAuthSalt = 'localhost'; 

/**
 * Installer: File Permission Configuration
 * 
 */
$config->chmodDir = 'localhost'; // permission for directories created by ProcessWire
$config->chmodFile = 'localhost'; // permission for files created by ProcessWire 

/**
 * Installer: Time zone setting
 * 
 */
$config->timezone = 'localhost';

/**
 * Installer: Admin theme
 * 
 */
$config->defaultAdminTheme = 'localhost';

/**
 * Installer: Unix timestamp of date/time installed
 * 
 * This is used to detect which when certain behaviors must be backwards compatible.
 * Please leave this value as-is.
 * 
 */
$config->installed = 1512033277;


/**
 * Installer: HTTP Hosts Whitelist
 * 
 */
$config->httpHosts = array('localhost', 'localhost');

as you can see im on windows + laragon. i used the latest dev version of pw

[flydev]

Hey @bernhard thanks for testing it !

2 minutes ago, bernhard said:

what is this single underscore function that you are using for translations?

I changed all the __() to _() before releasing the module, I don't know why, so I have no answer here lol I think it was a bit late for my brain. I will fix it in the next version.

 

5 minutes ago, bernhard said:

can you set the process module to be installed automatically by the main module please?

Yes I can, I was thinking in the first time to check if the ProcessModule is installed and if yes, it show the link to the manager, if not then no link appear. Why ?  because the ProcessModule is optional, you can configure Duplicator, the cron job and everything will run without using the manager.

But its just an idea.

 

9 minutes ago, bernhard said:

unfortunately i got an error on the last step:

I will test it under Windows, look like something weird is happening while adding the information to the config.php.

 

Thanks again for reporting theses issues !

[bernhard]

5 minutes ago, flydev said:

Yes I can, I was thinking in the first time to check if the ProcessModule is installed and if yes, it show the link to the manager, if not then no link appear. Why ?  because the ProcessModule is optional, you can configure Duplicator, the cron job and everything will run without using the manager.

But its just an idea.

Ah, I understand! Ok then a message would be nice to keep the first steps for everybody as easy as possible.

6 minutes ago, flydev said:

I will test it under Windows, look like something weird is happening while adding the information to the config.php.

Thanks!

[flydev]

I pushed a small update this night :

Fixed

• changed all _() call to __() for translation string
• the installer parse correctly the config.php file on windows machine
• small typo in ProcessDuplicator

Added

• check if ProcessDuplicator module is installed, if yes, then display the link in the local folder overview to the manager

 

I tested Duplicator on a Windows machine with PHP-5.6 and PHP-7.0, I could build a package and cloned the site without issue too.

 

[adrian]

Hi @flydev - thanks for all the hard work on this.

I am curious about the Dropbox API - are you planning on migrating to v2, or abandoning completely?

[flydev]

Its a good question, this night I tested the three different providers (Dropbox, Google and Amazon) and I read about Dropbox they dropped support for the v1 since September 2017. I was aware of that but I didn't knew the exact timeline.

So I removed Dropbox from the module in the version 1.0.2 but I found a small wrapper for the v2 coded by a guy on Github - already forked - I will test it if it can help to get back the Dropbox support on Duplicator. I also checked the migration guide, it should not be difficult to port the code to the v2. I just lack a bit time.

There is two things I wanted to implement asap (thanks @horst!), now there is three things

 

A small note: GoogleDrive and Amazon still work like a charm. Also I could tested yesterday a deployment on a Google Cloud VM instance (its a pain to setup, as always with Google) with the minimum resources, success!

[adrian]

Hey @flydev - I am finally getting around to trying this - everything works great, but I am wondering about triggering via a manual system cronjob. I expect Lazcron will result in quite a slowdown for whichever user is unlucky enough to trigger it. As for the PWCron module - I don't really like it at first glance because it looks like it is no longer being maintained.

[flydev]

Thanks for testing it @adrian! and I get what you mean.

Yes, LazyCron generally will slowdown, we can even say it will break completely the navigation on the website! and should be used with caution. Duplicator support LazyCron as "last" cron solution. About PwCron, I don't see objection for removing it from Duplicator or maybe it could stay as is for people who want to use it.

 

I added support for running cron jobs without external modules. Its already on Github (I will update the README).

 

You have to call it this way (/site/modules/Duplicator/cron.php) :

0 0 * * * php /www/mysite/wwwroot/site/modules/Duplicator/cron.php >/dev/null 2>&1

 

Edited December 5, 2017 by flydev
bad example