Jump to content
flydev

Duplicator: Backup and move sites

Recommended Posts

Now, the "package manager";

From this window, you can create or delete a package, and also download a packages from the "server backups folder" to your computer. But the best feature is that you can "sync" packages stored on a third-party provider. This mean that the module will check for existing packages on GoogleDrive, Dropbox or whatever (it depend on how you configured Duplicator) and list them so you can delete or download them to your computer, and deploy your nice website in a minute!

In the following example, I have packages on my Local Server, on GoogleDrive and Dropbox :

package_manager.thumb.png.21ed8fd749de1ebe9ae69f02d821ca52.png

 

  • Like 6

Share this post


Link to post
Share on other sites

As I am writing the documentation, if you guys have already some questions before I release the module, I will be happy to hear and include them in the FAQ :)

 

  • Like 3

Share this post


Link to post
Share on other sites

Hey, @flydev! I am sure those that are not beta testers will have most questions, so better ask us, eagerly waiting for the release, after it happens)
Glad you are back here, by the way!

  • Like 3

Share this post


Link to post
Share on other sites

@everyone : More beta-tester needed !

 

If you have a bit time, a local or staging web server - feel free to test it on a prod server - and you want to try this module, please send me a private message ! This module can't harm your server ;)

It can be tested on the following ProcessWire version:

  • 2.7.2 (no Composer here)
  • 2.8.x
  • 3.0.x

 

@Ivan Gretsky wanna test it ? :rolleyes:

 

  • Like 3

Share this post


Link to post
Share on other sites

I feel sorry again to make you guys waiting again, I got some works to finish asap, but theses tasks made me test Duplicator extensively and I think its ready for the release. It will not work for each hosting providers, but its a start.

Finally, it will come with a new feature: AutoDeploy.

With this feature, you will configure the FTP account and database informations, and the module will deploy automatically the web-site without having to run the installer manually.

  • Like 3

Share this post


Link to post
Share on other sites

Hi Flydev,

Thanks again, it looks very promising.

One question, would there be a way to have some sort of package profiles. I mean for example, may be I want to have a package profile that backups every day on the server excluding some large media files and an other package profile that backups everyweek with all the file on a S3 bucket.

Does that make sense (for me it does ;) ?

Regards

  • Like 1

Share this post


Link to post
Share on other sites
1 hour ago, antpre said:

Does that make sense (for me it does ;) ?

Yes it make sense, and if I remember correctly we already discussed through PM about this feature. Its not implemented but on the todo list now.

Share this post


Link to post
Share on other sites
26 minutes ago, flydev said:

I'll give you the url tonight ^-^

@flydev

I also would like to participate in testing if needed

  • Like 1

Share this post


Link to post
Share on other sites

@flydev

I'd like to test too. 

(Especially after I tried to set up backup/sync cron jobs via a server's crappy control panel where everything just works half-baked...)

  • Like 3

Share this post


Link to post
Share on other sites

@flydev : More beta-tester needed !

Hello
can I test your module? I have a lot of sites , someone heavy, so I can test and report :)

  • Like 2

Share this post


Link to post
Share on other sites

Duplicator-logo-v2-Dlong.thumb.png.f5822859fea5ced295c61d55869506fd.png

 

There you can download the module : https://github.com/flydev-fr/Duplicator

 

The documentation is incomplete, you have for now to explore the settings on your own. The install instructions are written but the SDKs are not uploaded yet, which mean that AmazonS3, Dropbox and GoogleDrive are only available by installing them through composer. I will upload them tomorrow but they are not recently tested.

Anyway, there is no requirements to run Duplicator as is. Just install Duplicator and Duplicator - Packages Manager then go to Setup > Duplicator and build your package by clicking the button. If everything run smooth, you will see the package listed and you will be able to download it directly and also the installer.

To restore a website, just upload the package and the installer then go to yourwebsite.com/installer.php then follow the instructions.

PLEASE, do not forget - after the end of the process - to delete the installer.php file and your package (this will be fixed in the v0.0.45).

 

It should work with default theme and the UIkit theme.

A dedicated thread will be created in the modules forum soon. Do not hesitate to share here your results by providing your hosting type / provider / config, etc. :)

Edited by flydev
logo (made by @szabesz)
  • Like 6

Share this post


Link to post
Share on other sites

@flydev Thank you so much for this module! Just a side-note: please consider turning the next version number to at least 0.1.0  or better yet 1.0.0 (I do not think this module is in beta state as it is stable. It might lack some functions you have not yet found the time to finish but those can be pointed out in the read me and not in the version number :) ) We have a recent discussion about this and it seems to be beneficial to use semver 2.0 : https://processwire.com/talk/topic/17767-module-release-notes/?do=findComment&comment=156350

 

And for those who doesn't already know I would like to point out that Duplicator is not just about duplicating but making regular backups (via (lazy)cron).

Edited by szabesz
typo
  • Like 2

Share this post


Link to post
Share on other sites

hi flydev

thanks for releasing this :)

i just tried it and i got multiple errors "call to undefined function _(...". i changed all occurences from _( to __( and it worked. what is this single underscore function that you are using for translations? same for the process module.

also i get an "unrecognized path" for http://duplicator.dev/processwire/setup/Duplicator/?action=packages

can you set the process module to be installed automatically by the main module please? :) 

the backup worked like a charm. the installer seems to be quite similar to my kickstart project. i think there is great potential to combine both tools!

unfortunately i got an error on the last step:

5a1fcf6b356b7_2017-11-3010_27_51-Duplicatorinstaller.png.c671ebfda608327c01e8fff9642c0c59.png

the problem seems to be the config.php

/**
 * Installer: Database Configuration
 * 
 */
$config->dbHost = 'localhost';
$config->dbName = 'localhost';
$config->dbUser = 'localhost';
$config->dbPass = 'localhost';
$config->dbPort = 'localhost';

/**
 * Installer: User Authentication Salt 
 * 
 * Must be retained if you migrate your site from one server to another
 * 
 */
$config->userAuthSalt = 'localhost'; 

/**
 * Installer: File Permission Configuration
 * 
 */
$config->chmodDir = 'localhost'; // permission for directories created by ProcessWire
$config->chmodFile = 'localhost'; // permission for files created by ProcessWire 

/**
 * Installer: Time zone setting
 * 
 */
$config->timezone = 'localhost';

/**
 * Installer: Admin theme
 * 
 */
$config->defaultAdminTheme = 'localhost';

/**
 * Installer: Unix timestamp of date/time installed
 * 
 * This is used to detect which when certain behaviors must be backwards compatible.
 * Please leave this value as-is.
 * 
 */
$config->installed = 1512033277;


/**
 * Installer: HTTP Hosts Whitelist
 * 
 */
$config->httpHosts = array('localhost', 'localhost');

as you can see im on windows + laragon. i used the latest dev version of pw :)

  • Like 1

Share this post


Link to post
Share on other sites

Hey @bernhard thanks for testing it !

2 minutes ago, bernhard said:

what is this single underscore function that you are using for translations?

I changed all the __() to _() before releasing the module, I don't know why, so I have no answer here lol I think it was a bit late for my brain. I will fix it in the next version.

 

5 minutes ago, bernhard said:

can you set the process module to be installed automatically by the main module please?

Yes I can, I was thinking in the first time to check if the ProcessModule is installed and if yes, it show the link to the manager, if not then no link appear. Why ?  because the ProcessModule is optional, you can configure Duplicator, the cron job and everything will run without using the manager.

But its just an idea.

 

9 minutes ago, bernhard said:

unfortunately i got an error on the last step:

I will test it under Windows, look like something weird is happening while adding the information to the config.php.

 

Thanks again for reporting theses issues !

  • Like 2

Share this post


Link to post
Share on other sites
5 minutes ago, flydev said:

Yes I can, I was thinking in the first time to check if the ProcessModule is installed and if yes, it show the link to the manager, if not then no link appear. Why ?  because the ProcessModule is optional, you can configure Duplicator, the cron job and everything will run without using the manager.

But its just an idea.

Ah, I understand! Ok then a message would be nice to keep the first steps for everybody as easy as possible. :)

6 minutes ago, flydev said:

I will test it under Windows, look like something weird is happening while adding the information to the config.php.

Thanks!

  • Like 1

Share this post


Link to post
Share on other sites

I pushed a small update this night :

Fixed

  • changed all _() call to __() for translation string
  • the installer parse correctly the config.php file on windows machine
  • small typo in ProcessDuplicator

Added

  • check if ProcessDuplicator module is installed, if yes, then display the link in the local folder overview to the manager

 

I tested Duplicator on a Windows machine with PHP-5.6 and PHP-7.0, I could build a package and cloned the site without issue too.

 

  • Like 4

Share this post


Link to post
Share on other sites

Hi @flydev - thanks for all the hard work on this.

I am curious about the Dropbox API - are you planning on migrating to v2, or abandoning completely?

  • Like 1

Share this post


Link to post
Share on other sites

Its a good question, this night I tested the three different providers (Dropbox, Google and Amazon) and I read about Dropbox they dropped support for the v1 since September 2017. I was aware of that but I didn't knew the exact timeline.

So I removed Dropbox from the module in the version 1.0.2 but I found a small wrapper for the v2 coded by a guy on Github - already forked - I will test it if it can help to get back the Dropbox support on Duplicator. I also checked the migration guide, it should not be difficult to port the code to the v2. I just lack a bit time.

There is two things I wanted to implement asap (thanks @horst!), now there is three things :lol:

 

A small note: GoogleDrive and Amazon still work like a charm. Also I could tested yesterday a deployment on a Google Cloud VM instance (its a pain to setup, as always with Google) with the minimum resources, success!

  • Like 3

Share this post


Link to post
Share on other sites

Hey @flydev - I am finally getting around to trying this - everything works great, but I am wondering about triggering via a manual system cronjob. I expect Lazcron will result in quite a slowdown for whichever user is unlucky enough to trigger it. As for the PWCron module - I don't really like it at first glance because it looks like it is no longer being maintained.

  • Like 3

Share this post


Link to post
Share on other sites

Thanks for testing it @adrian! and I get what you mean.

Yes, LazyCron generally will slowdown, we can even say it will break completely the navigation on the website! and should be used with caution. Duplicator support LazyCron as "last" cron solution. About PwCron, I don't see objection for removing it from Duplicator or maybe it could stay as is for people who want to use it.

 

I added support for running cron jobs without external modules. Its already on Github (I will update the README).

 

You have to call it this way (/site/modules/Duplicator/cron.php) :

0 0 * * * php /www/mysite/wwwroot/site/modules/Duplicator/cron.php >/dev/null 2>&1

 

Edited by flydev
bad example
  • Like 4
  • Thanks 1

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By jaro
      This module (github) does with site/assets/files what Ryan's DatabaseBackups module does with the database:
      Backup site/assets Download ZIP archive Upload ZIP archive Restore site/assets Motivation: This module can be the missing part for projects with content backup responsibility on the client's side: The client will be able to download DB and assets/files snapshots through the backend without filesystem access, thus backing up all content themselves.
      Release state alpha – do not use in production environments.
      Credits for the nice UI go to @ryan – I reused most of it and some other code from the DatabaseBackups module.
    • By David Karich
      ProcessWire InputfieldRepeaterMatrixDuplicate
      Thanks to the great ProModule "RepeaterMatrix" I have the possibility to create complex repeater items. With it I have created a quite powerful page builder. Many different content modules, with many more possible design options. The RepeaterMatrix module supports the cloning of items, but only within the same page. Now I often have the case that very design-intensive pages and items are created. If you want to use a content module on a different page (e.g. in the same design), you have to rebuild each item manually every time.
      This module extends the commercial ProModule "RepeaterMatrix" by the function to duplicate repeater items from one page to another page. The condition is that the target field is the same matrix field from which the item is duplicated. This module is currently understood as proof of concept. There are a few limitations that need to be considered. The intention of the module is that this functionality is integrated into the core of RepeaterMatrix and does not require an extra module.
      Check out the screencast
      What the module can do
      Duplicate a repeater item from one page to another No matter how complex the item is Full support for file and image fields Multilingual support Support of Min and Max settings Live synchronization of clipboard between multiple browser tabs. Copy an item and simply switch the browser tab to the target page and you will immediately see the past button Support of multiple RepeaterMatrix fields on one page Configurable which roles and fields are excluded Duplicated items are automatically pasted to the end of the target field and set to hidden status so that changes are not directly published Automatic clipboard update when other items are picked Automatically removes old clipboard data if it is not pasted within 6 hours Delete clipboard itself by clicking the selected item again Benefit: unbelievably fast workflow and content replication What the module can't do
      Before an item can be duplicated in its current version, the source page must be saved. This means that if you make changes to an item and copy this, the old saved state will be duplicated Dynamic loading is currently not possible. Means no AJAX. When pasting, the target page is saved completely No support for nested repeater items. Currently only first level items can be duplicated. Means a repeater field in a repeater field cannot be duplicated. Workaround: simply duplicate the parent item Dynamic reloading and adding of repeater items cannot be registered. Several interfaces and events from the core are missing. The initialization occurs only once after the page load event Changelog
      1.0.4
      Bug fix: Various bug fixes and improvements in live synchronization Bug fix: Items are no longer inserted when the normal save button is clicked. Only when the past button is explicitly clicked Feature: Support of multiple repeater fields in one page Feature: Support of repeater Min/Max settings Feature: Configurable roles and fields Enhancement: Improved clipboard management Enhancement: Documentation improvement Enhancement: Corrected few typos #1 1.0.3
      Feature: Live synchronization Enhancement: Load the module only in the backend Enhancement: Documentation improvement 1.0.2
      Bug fix: Various bug fixes and improvements in JS functions Enhancement: Documentation improvement Enhancement: Corrected few typos 1.0.1
      Bug fix: Various bug fixes and improvements in the duplication process 1.0.0
      Initial release Support this module
      If this module is useful for you, I am very thankful for your small donation: Donate 5,- Euro (via PayPal – or an amount of your choice. Thank you!)
      Download this module
      > Github: https://github.com/FlipZoomMedia/InputfieldRepeaterMatrixDuplicate
      > PW module directory: – soon –
    • By NorbertH
      Is there a hook to do something right after cloning a page ?
      I want the page to be saved right after cloning it either from the button in the tree or from a lister, because saving the page triggers several calculations that are not triggered by just cloning the page.
       
      Thanks in advance !
    • By John W.
      Question 1
      I recently installed PW 3.0.62 for a new site and also have sites running older version of PW 3.x.
      Can I export the database on an older version of PW 3.x and import it to PW 3.0.62 without any issues?
       
      Question 2
      (This is kind of alternative to the above for long term use - and maybe a better solution...)
      On  the sites I've previously built I have templates (home, basic-page, contact) and fields that I commonly use, such as business_name, phone_1.  The last site I built is running PW 3.0.42.  I was considering cloning this into a local site and running the upgrade module to bring it up to PW 3.0.62. From there on out when I start I new project I could just run the PW upgrade module, copy the folder to the location for the new project and duplicate the database using the new projects name.

      So basically, I'll always keep a "blank slate" site that I can just run the PW upgrade on, then duplicate into a new project. This would cut down on the work and time spent having to re-create these common fields, that I use. From there, I would just add fields, templates, etc, specific for the new website project.

      Is this a sound approach to speed up development?

       
×
×
  • Create New...