flydev

Duplicator: Backup and move sites

Recommended Posts

Now, the "package manager";

From this window, you can create or delete a package, and also download a packages from the "server backups folder" to your computer. But the best feature is that you can "sync" packages stored on a third-party provider. This mean that the module will check for existing packages on GoogleDrive, Dropbox or whatever (it depend on how you configured Duplicator) and list them so you can delete or download them to your computer, and deploy your nice website in a minute!

In the following example, I have packages on my Local Server, on GoogleDrive and Dropbox :

package_manager.thumb.png.21ed8fd749de1ebe9ae69f02d821ca52.png

 

  • Like 6

Share this post


Link to post
Share on other sites

As I am writing the documentation, if you guys have already some questions before I release the module, I will be happy to hear and include them in the FAQ :)

 

  • Like 3

Share this post


Link to post
Share on other sites

Hey, @flydev! I am sure those that are not beta testers will have most questions, so better ask us, eagerly waiting for the release, after it happens)
Glad you are back here, by the way!

  • Like 3

Share this post


Link to post
Share on other sites

@everyone : More beta-tester needed !

 

If you have a bit time, a local or staging web server - feel free to test it on a prod server - and you want to try this module, please send me a private message ! This module can't harm your server ;)

It can be tested on the following ProcessWire version:

  • 2.7.2 (no Composer here)
  • 2.8.x
  • 3.0.x

 

@Ivan Gretsky wanna test it ? :rolleyes:

 

  • Like 3

Share this post


Link to post
Share on other sites

I feel sorry again to make you guys waiting again, I got some works to finish asap, but theses tasks made me test Duplicator extensively and I think its ready for the release. It will not work for each hosting providers, but its a start.

Finally, it will come with a new feature: AutoDeploy.

With this feature, you will configure the FTP account and database informations, and the module will deploy automatically the web-site without having to run the installer manually.

  • Like 3

Share this post


Link to post
Share on other sites

Hi Flydev,

Thanks again, it looks very promising.

One question, would there be a way to have some sort of package profiles. I mean for example, may be I want to have a package profile that backups every day on the server excluding some large media files and an other package profile that backups everyweek with all the file on a S3 bucket.

Does that make sense (for me it does ;) ?

Regards

  • Like 1

Share this post


Link to post
Share on other sites
1 hour ago, antpre said:

Does that make sense (for me it does ;) ?

Yes it make sense, and if I remember correctly we already discussed through PM about this feature. Its not implemented but on the todo list now.

Share this post


Link to post
Share on other sites
26 minutes ago, flydev said:

I'll give you the url tonight ^-^

@flydev

I also would like to participate in testing if needed

  • Like 1

Share this post


Link to post
Share on other sites

@flydev

I'd like to test too. 

(Especially after I tried to set up backup/sync cron jobs via a server's crappy control panel where everything just works half-baked...)

  • Like 3

Share this post


Link to post
Share on other sites

@flydev : More beta-tester needed !

Hello
can I test your module? I have a lot of sites , someone heavy, so I can test and report :)

  • Like 2

Share this post


Link to post
Share on other sites

Duplicator-logo-v2-Dlong.thumb.png.f5822859fea5ced295c61d55869506fd.png

 

There you can download the module : https://github.com/flydev-fr/Duplicator

 

The documentation is incomplete, you have for now to explore the settings on your own. The install instructions are written but the SDKs are not uploaded yet, which mean that AmazonS3, Dropbox and GoogleDrive are only available by installing them through composer. I will upload them tomorrow but they are not recently tested.

Anyway, there is no requirements to run Duplicator as is. Just install Duplicator and Duplicator - Packages Manager then go to Setup > Duplicator and build your package by clicking the button. If everything run smooth, you will see the package listed and you will be able to download it directly and also the installer.

To restore a website, just upload the package and the installer then go to yourwebsite.com/installer.php then follow the instructions.

PLEASE, do not forget - after the end of the process - to delete the installer.php file and your package (this will be fixed in the v0.0.45).

 

It should work with default theme and the UIkit theme.

A dedicated thread will be created in the modules forum soon. Do not hesitate to share here your results by providing your hosting type / provider / config, etc. :)

Edited by flydev
logo (made by @szabesz)
  • Like 6

Share this post


Link to post
Share on other sites

@flydev Thank you so much for this module! Just a side-note: please consider turning the next version number to at least 0.1.0  or better yet 1.0.0 (I do not think this module is in beta state as it is stable. It might lack some functions you have not yet found the time to finish but those can be pointed out in the read me and not in the version number :) ) We have a recent discussion about this and it seems to be beneficial to use semver 2.0 : https://processwire.com/talk/topic/17767-module-release-notes/?do=findComment&comment=156350

 

And for those who doesn't already know I would like to point out that Duplicator is not just about duplicating but making regular backups (via (lazy)cron).

Edited by szabesz
typo
  • Like 2

Share this post


Link to post
Share on other sites

hi flydev

thanks for releasing this :)

i just tried it and i got multiple errors "call to undefined function _(...". i changed all occurences from _( to __( and it worked. what is this single underscore function that you are using for translations? same for the process module.

also i get an "unrecognized path" for http://duplicator.dev/processwire/setup/Duplicator/?action=packages

can you set the process module to be installed automatically by the main module please? :) 

the backup worked like a charm. the installer seems to be quite similar to my kickstart project. i think there is great potential to combine both tools!

unfortunately i got an error on the last step:

5a1fcf6b356b7_2017-11-3010_27_51-Duplicatorinstaller.png.c671ebfda608327c01e8fff9642c0c59.png

the problem seems to be the config.php

/**
 * Installer: Database Configuration
 * 
 */
$config->dbHost = 'localhost';
$config->dbName = 'localhost';
$config->dbUser = 'localhost';
$config->dbPass = 'localhost';
$config->dbPort = 'localhost';

/**
 * Installer: User Authentication Salt 
 * 
 * Must be retained if you migrate your site from one server to another
 * 
 */
$config->userAuthSalt = 'localhost'; 

/**
 * Installer: File Permission Configuration
 * 
 */
$config->chmodDir = 'localhost'; // permission for directories created by ProcessWire
$config->chmodFile = 'localhost'; // permission for files created by ProcessWire 

/**
 * Installer: Time zone setting
 * 
 */
$config->timezone = 'localhost';

/**
 * Installer: Admin theme
 * 
 */
$config->defaultAdminTheme = 'localhost';

/**
 * Installer: Unix timestamp of date/time installed
 * 
 * This is used to detect which when certain behaviors must be backwards compatible.
 * Please leave this value as-is.
 * 
 */
$config->installed = 1512033277;


/**
 * Installer: HTTP Hosts Whitelist
 * 
 */
$config->httpHosts = array('localhost', 'localhost');

as you can see im on windows + laragon. i used the latest dev version of pw :)

  • Like 1

Share this post


Link to post
Share on other sites

Hey @bernhard thanks for testing it !

2 minutes ago, bernhard said:

what is this single underscore function that you are using for translations?

I changed all the __() to _() before releasing the module, I don't know why, so I have no answer here lol I think it was a bit late for my brain. I will fix it in the next version.

 

5 minutes ago, bernhard said:

can you set the process module to be installed automatically by the main module please?

Yes I can, I was thinking in the first time to check if the ProcessModule is installed and if yes, it show the link to the manager, if not then no link appear. Why ?  because the ProcessModule is optional, you can configure Duplicator, the cron job and everything will run without using the manager.

But its just an idea.

 

9 minutes ago, bernhard said:

unfortunately i got an error on the last step:

I will test it under Windows, look like something weird is happening while adding the information to the config.php.

 

Thanks again for reporting theses issues !

  • Like 2

Share this post


Link to post
Share on other sites
5 minutes ago, flydev said:

Yes I can, I was thinking in the first time to check if the ProcessModule is installed and if yes, it show the link to the manager, if not then no link appear. Why ?  because the ProcessModule is optional, you can configure Duplicator, the cron job and everything will run without using the manager.

But its just an idea.

Ah, I understand! Ok then a message would be nice to keep the first steps for everybody as easy as possible. :)

6 minutes ago, flydev said:

I will test it under Windows, look like something weird is happening while adding the information to the config.php.

Thanks!

  • Like 1

Share this post


Link to post
Share on other sites

I pushed a small update this night :

Fixed

  • changed all _() call to __() for translation string
  • the installer parse correctly the config.php file on windows machine
  • small typo in ProcessDuplicator

Added

  • check if ProcessDuplicator module is installed, if yes, then display the link in the local folder overview to the manager

 

I tested Duplicator on a Windows machine with PHP-5.6 and PHP-7.0, I could build a package and cloned the site without issue too.

 

  • Like 4

Share this post


Link to post
Share on other sites

Hi @flydev - thanks for all the hard work on this.

I am curious about the Dropbox API - are you planning on migrating to v2, or abandoning completely?

  • Like 1

Share this post


Link to post
Share on other sites

Its a good question, this night I tested the three different providers (Dropbox, Google and Amazon) and I read about Dropbox they dropped support for the v1 since September 2017. I was aware of that but I didn't knew the exact timeline.

So I removed Dropbox from the module in the version 1.0.2 but I found a small wrapper for the v2 coded by a guy on Github - already forked - I will test it if it can help to get back the Dropbox support on Duplicator. I also checked the migration guide, it should not be difficult to port the code to the v2. I just lack a bit time.

There is two things I wanted to implement asap (thanks @horst!), now there is three things :lol:

 

A small note: GoogleDrive and Amazon still work like a charm. Also I could tested yesterday a deployment on a Google Cloud VM instance (its a pain to setup, as always with Google) with the minimum resources, success!

  • Like 3

Share this post


Link to post
Share on other sites

Hey @flydev - I am finally getting around to trying this - everything works great, but I am wondering about triggering via a manual system cronjob. I expect Lazcron will result in quite a slowdown for whichever user is unlucky enough to trigger it. As for the PWCron module - I don't really like it at first glance because it looks like it is no longer being maintained.

  • Like 3

Share this post


Link to post
Share on other sites

Thanks for testing it @adrian! and I get what you mean.

Yes, LazyCron generally will slowdown, we can even say it will break completely the navigation on the website! and should be used with caution. Duplicator support LazyCron as "last" cron solution. About PwCron, I don't see objection for removing it from Duplicator or maybe it could stay as is for people who want to use it.

 

I added support for running cron jobs without external modules. Its already on Github (I will update the README).

 

You have to call it this way (/site/modules/Duplicator/cron.php) :

0 0 * * * php /www/mysite/wwwroot/site/modules/Duplicator/cron.php >/dev/null 2>&1

 

Edited by flydev
bad example
  • Like 4
  • Thanks 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now


  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By holmescreek
      Question 1
      I recently installed PW 3.0.62 for a new site and also have sites running older version of PW 3.x.
      Can I export the database on an older version of PW 3.x and import it to PW 3.0.62 without any issues?
       
      Question 2
      (This is kind of alternative to the above for long term use - and maybe a better solution...)
      On  the sites I've previously built I have templates (home, basic-page, contact) and fields that I commonly use, such as business_name, phone_1.  The last site I built is running PW 3.0.42.  I was considering cloning this into a local site and running the upgrade module to bring it up to PW 3.0.62. From there on out when I start I new project I could just run the PW upgrade module, copy the folder to the location for the new project and duplicate the database using the new projects name.

      So basically, I'll always keep a "blank slate" site that I can just run the PW upgrade on, then duplicate into a new project. This would cut down on the work and time spent having to re-create these common fields, that I use. From there, I would just add fields, templates, etc, specific for the new website project.

      Is this a sound approach to speed up development?

       
    • By holmescreek
      I just posted a question about moving PW from a resting domain to a production domain, after consideration I figure I would create a new post for a second question I have.
      I noticed the coming feature of client size image resizing. This will be really helpful, as I have some clients on a Satellite connection. I'm telling you, them trying to upload 15MB images, or them getting them straight on batch resizers (or Photoshop) to resize before upload is a real chore. Needless to say, Ryan has been doing an excellent job addressing this.
      My question, is that I have several clients on PW 2.x thru 3.x. Is there a really good guide on steps that I need to take to update the core to take advantage of the features of new releases? Are their perils to look out for?
      Yeah, after developing several sites, I feel a bit silly about asking this one, but, the docs doesn't seem to be clear in a "general" sense of upgrading the core across all versions.
      Thanks everyone for your time and assistance.
    • By suntrop
      Hi all
      I have big troubles with my DB backups and phpMyAdmin. I can't manage to import a backup.
      The phpMyAdmin export settings are left to the defaults. The error message I get:
      #1064 - You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'INSERT INTO `caches` (`name`, `data`, `expires`) VALUES ('ModulesUninstalled.in' at line 11 When I delete the complete 'cache' section there are almost for all tables the same errors. (part of the dump attached)
      I had this problem before and back then (couple months ago) I think I copied almost every table individually. I thought it was specific to that DB. But I can't do a dump/import of any of my PW installations.
      MySQL: 5.6.27
      PHP version: 7.0.11
      phpMyAdmin: 4.5.5.1
      Anybody knows that problem? Or what to do?
      phpMyAdmin SQL Dump.sql
    • By vwatson
      Is it possible to add check boxes in order to select multiple backup files and delete them all at once, rather than having to click each one, then Delete?
      Also it would be helpful to make the Restore process more distinct from Deleting.  I must have inadvertently clicked the wrong button once and restored a backup unintentionally, wiping out a day's work. My mistake of course. I obviously wasn't paying close enough attention.  
    • By Fantomas
      Hi all,
      I'm still new to processwire and now I have to create the first hook.
      I need to modify the page-clone module, so there is a field in pages that has to be empty after cloning the page. How could I start this task?