Jump to content
flydev

Duplicator: Backup and move sites

Recommended Posts

I have encountered an issue for Duplicator ver 1.2.9.

I am moving a site from localhost(windows 10) to Amazon Linux. I use RDS, which is a standalone server for database, so the dbHost is not localhost.

The issue happened in the installation process after entering the database data and proceed. There's an error telling config file cannot be found. Then I go to config.php, there are at least 4 places are using old information. They are dbHost, dbName, dbUser, dbPass and httpHosts. I think for some reasons the config.php did not update at all after the installation complete. After I manually update the fields, the site works fine now. This means database import is working fine. I wonder if it is the dbHost matters. It is a long URL ended with xxx.rds.amazonaws.com.

  • Like 3

Share this post


Link to post
Share on other sites

Hi @Karl_T sorry for that and thanks for reporting this issue, I think I have an idea why is happening - checking this afternoon.

  • Like 4
  • Thanks 1

Share this post


Link to post
Share on other sites

Hi

Latest Duplicator does not work here.

It does not include ./wire and other stuff.

Version 1.1.7 works.

Thank you.

  • Thanks 1

Share this post


Link to post
Share on other sites

I am running version 1.2.9 without issues on two sites.

13 minutes ago, theo said:

It does not include ./wire and other stuff.

You might want to double check and re-save your settings. There might have been changes regarding the way the module stores what to include and what not to.

  • Like 2

Share this post


Link to post
Share on other sites

If duplicator fails there is no sweat at all. Simply upload your site folder and database.
I never got it why duplicator has an advantage over manually uploading your site folder and database.
The manual down and upload is compatible with all versions of processwire :)
I also don't see any time saving when using it.

Share this post


Link to post
Share on other sites
43 minutes ago, pwired said:

I never got it why duplicator has an advantage over manually uploading your site folder and database.

It all depends on your workflow. Also, Duplicator is not just about cloning a site but for creating backups. Normally I use it for daily backups and/or for quick backups before "performing sensitive changes", just to be on the safe side...

  • Like 4

Share this post


Link to post
Share on other sites
Quote

It all depends on your workflow. Also, Duplicator is not just about cloning a site but for creating backups. Normally I use it for daily backups and/or for quick backups before "performing sensitive changes", just to be on the safe side...

Maybe I still haven't enough websites on my shoulders to see the use of it, so I should install and evaluate it. What about scripting your cpanel over the web ? Does anyone know if there exists a scripting language for automating cpanel tasks ? I don't mean cronjobs but something like cUrl.

  • Like 2

Share this post


Link to post
Share on other sites
7 minutes ago, pwired said:

What about scripting your cpanel over the web ? Does anyone know if there exists a scripting language for automating cpanel tasks ? I don't mean cronjobs but something like cUrl.

I always insist on having SSH no matter what because BASH is also in my toolset. I either clone a complete site with my own script or just the database. I make admin changes in production (if I'm not sure about the results in advance, then I perform tests in dev first) and after that I use my cloning script to replace the dev database from prod which takes only a few (4-7) seconds to do if one does not have to deal with a huge dbs (which I currently don't). 

https://processwire.com/talk/topic/14837-best-practise-to-implement-changes-in-pw-from-development-to-production/?do=findComment&comment=133726

https://processwire.com/talk/topic/14837-best-practise-to-implement-changes-in-pw-from-development-to-production/?do=findComment&comment=133741

 

 

  • Like 1

Share this post


Link to post
Share on other sites
Quote

   SSH . . .

That makes me thinking about writing your own scripts that drive cPanel SSH Commands.
Maybe with AutoHotKey for windows. That would be a cool way to automate and task schedule
backups, down and uploads.

  • Like 1

Share this post


Link to post
Share on other sites
3 hours ago, szabesz said:

I am running version 1.2.9 without issues on two sites.

You might want to double check and re-save your settings. There might have been changes regarding the way the module stores what to include and what not to.

I have completely uninstalled and deleted it, even checked in the db for the module entry, then downloaded and installed the latest version (1.2.9) and no, it does not work as before.

The wire folder is not included etc.

The package is only 2.11 MB instead of more than 20 MB before.

Thank you.

  • Like 2

Share this post


Link to post
Share on other sites

@szabesz Probably you remember my "special" setup.

I had some problems before 1.1.6 in the area of mod_userdir and this:

 

 

Maybe the current problem is related?

  • Like 2

Share this post


Link to post
Share on other sites

OK, I had a quick look at the source of Duplicator.module

On line 640, there is a "isWritable" check:

if(!$f->isReadable() || !$f->isWritable()) { continue; }

Why does the file to copy have to be writable?

My "wire" folder is not writable by "wwwrun" because it is not necessary, it has to be readable though.

If I remove this check, it works as before.

Thank you.

  • Like 1
  • Thanks 1

Share this post


Link to post
Share on other sites

There is another issue in my case:

The parent folder is included in v 1.2.9.

Probably has to do with mod_userdir.

So it does not work because it is extracting everything to /parentfolder/wire etc.

As the above issue, this was all working in v 1.1.7

  • Like 1

Share this post


Link to post
Share on other sites

Not sure if it's all servers, but I am finding the downloading a package from Setup > Duplicator is resulting in partial downloads. I don't have time to investigate at the moment, but just a heads up in case someone is trusting the download to be complete.

  • Like 2

Share this post


Link to post
Share on other sites
14 minutes ago, adrian said:

Setup > Duplicator is resulting in partial downloads. I don't have time to investigate at the moment, but just a heads up in case someone is trusting the download to be complete.

I guess it depends on the file size of the ZIP file. Big files cannot be downloaded for example if not enough memory is allocated for the process. I always download the ZIP files via FTP which is reliable in my case.

  • Like 1

Share this post


Link to post
Share on other sites
On 5/2/2018 at 10:48 AM, theo said:

Why does the file to copy have to be writable?

My "wire" folder is not writable by "wwwrun" because it is not necessary, it has to be readable though. 

@theo The new function is based on the official WireFileTools::zip() function and I didn't saw this check. Nice catch 👍

 

On 5/2/2018 at 5:19 PM, theo said:

There is another issue in my case:

The parent folder is included in v 1.2.9.

Probably has to do with mod_userdir.

It has not do with the issue I remember. I just checked and its a silly mistake I made while updating the module. Thanks again @theo . It will be fixed in Duplicator-1.2.10

 

On 5/2/2018 at 9:32 PM, adrian said:

Not sure if it's all servers, but I am finding the downloading a package from Setup > Duplicator is resulting in partial downloads. I don't have time to investigate at the moment, but just a heads up in case someone is trusting the download to be complete.

I never ran into into this issue but it look like it can happen. I have a site with 600MB stuff and his database size at 1GB+, I have no timeout building the package or downloading issue. To me, it look like a server settings.

If you can, please send me the server details so I can try to reproduce it.

PS: still didn't found the time to check what is going on GoogleDrive.

 

@szabesz any feedback about Dropbox v2 ? 😃

 

  • Like 4

Share this post


Link to post
Share on other sites
1 hour ago, flydev said:

any feedback about Dropbox v2 ?

I do not have too much extra time these days so I postponed testing it, also because you mentioned possible idrive support and I thought I would wait for that too, I was thinking about testing both "parallel".

  • Like 1

Share this post


Link to post
Share on other sites

A small news about Duplicator and why its not updated recently:

On the current state of the module, it is usable for small website, I mean small database (I am aware of the issue which include the  parent directory in the compressed package which make the package non deployable by the installer).

As stated by Ryan itself, If you are working with really large files, it may be better to perform that task directly in PhpMyAdmin or the native MySQL client.

And as I am working now with giga-sized databases, its the time to rewrite this module. So what is planned ? 

 

Duplicator will be made on top of PHPBU in the hope to provide to the community a professional focused backup service for your websites.

Quote

PHPBU is a php framework that creates and encrypts backups and moves your backups to other servers or cloud services
and assists you monitor your backup creation

More information :

 

Any suggestion welcome. Have a nice day ! ✌️

  • Like 10
  • Thanks 1

Share this post


Link to post
Share on other sites
On 6/21/2018 at 11:04 AM, flydev said:

Any suggestion welcome. Have a nice day ! ✌️

This sounds awesome! I can offer my humble testing skills even though I'm not having too much free time these days but this project is more important than ever (thinking of GDPR compliant backups, ie. encryption for backups).

Last time Flydev could not reach out to too many active testers, so now I encourage anyone reading these lines to join in and offer testing – maybe even development – help to Flydev in advance if you can really do it when he needs it.

Also, please do not just hit the like button but reply with suggestions too 😉 BTW, my first suggestions is: besides being GUI friendly, the new module could also be cli friendly: meaning wireshell integration. 

  • Like 2
  • Thanks 1

Share this post


Link to post
Share on other sites
On 6/25/2018 at 3:15 PM, arjen said:

I'm in 🙂

I'm too.

  • Like 3

Share this post


Link to post
Share on other sites

Quick question, the automatic backups through LazyCron aren't working for me. Manual backups are working fine (both locally and with remote FTP backup), but the automatic backups won't trigger at all.

Here are my settings:

  • Backup mode: LazyCron - Triggered by a pageview (installed)
  • Event trigger: 1 hour
  • Remove backups older than: 1 year

I have waited for several hours, the site had traffic during that time, and I also visited some pages as a logged-in administrator and as an anonymous visitor. But no backups occured, there are also no log entries at all, so it doesn't look like it just bugged out. Do I need to configure anything else? LazyCron is installed, it doesn't have any module options though. Or do I need to manually call the LazyCron hooks in ready.php or something like that? Not sure what I am missing.

Thanks!

  • Like 1

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By David Karich
      ProcessWire InputfieldRepeaterMatrixDuplicate
      Thanks to the great ProModule "RepeaterMatrix" I have the possibility to create complex repeater items. With it I have created a quite powerful page builder. Many different content modules, with many more possible design options. The RepeaterMatrix module supports the cloning of items, but only within the same page. Now I often have the case that very design-intensive pages and items are created. If you want to use a content module on a different page (e.g. in the same design), you have to rebuild each item manually every time.
      This module extends the commercial ProModule "RepeaterMatrix" by the function to duplicate repeater items from one page to another page. The condition is that the target field is the same matrix field from which the item is duplicated. This module is currently understood as proof of concept. There are a few limitations that need to be considered. The intention of the module is that this functionality is integrated into the core of RepeaterMatrix and does not require an extra module.
      Check out the screencast
      What the module can do
      Duplicate a repeater item from one page to another No matter how complex the item is Full support for file and image fields Multilingual support Support of Min and Max settings Live synchronization of clipboard between multiple browser tabs. Copy an item and simply switch the browser tab to the target page and you will immediately see the past button Support of multiple RepeaterMatrix fields on one page Configurable which roles and fields are excluded Duplicated items are automatically pasted to the end of the target field and set to hidden status so that changes are not directly published Automatic clipboard update when other items are picked Automatically removes old clipboard data if it is not pasted within 6 hours Delete clipboard itself by clicking the selected item again Benefit: unbelievably fast workflow and content replication What the module can't do
      Before an item can be duplicated in its current version, the source page must be saved. This means that if you make changes to an item and copy this, the old saved state will be duplicated Dynamic loading is currently not possible. Means no AJAX. When pasting, the target page is saved completely No support for nested repeater items. Currently only first level items can be duplicated. Means a repeater field in a repeater field cannot be duplicated. Workaround: simply duplicate the parent item Dynamic reloading and adding of repeater items cannot be registered. Several interfaces and events from the core are missing. The initialization occurs only once after the page load event Changelog
      1.0.4
      Bug fix: Various bug fixes and improvements in live synchronization Bug fix: Items are no longer inserted when the normal save button is clicked. Only when the past button is explicitly clicked Feature: Support of multiple repeater fields in one page Feature: Support of repeater Min/Max settings Feature: Configurable roles and fields Enhancement: Improved clipboard management Enhancement: Documentation improvement Enhancement: Corrected few typos #1 1.0.3
      Feature: Live synchronization Enhancement: Load the module only in the backend Enhancement: Documentation improvement 1.0.2
      Bug fix: Various bug fixes and improvements in JS functions Enhancement: Documentation improvement Enhancement: Corrected few typos 1.0.1
      Bug fix: Various bug fixes and improvements in the duplication process 1.0.0
      Initial release Support this module
      If this module is useful for you, I am very thankful for your small donation: Donate 5,- Euro (via PayPal – or an amount of your choice. Thank you!)
      Download this module
      > Github: https://github.com/FlipZoomMedia/InputfieldRepeaterMatrixDuplicate
      > PW module directory: https://modules.processwire.com/modules/inputfield-repeater-matrix-duplicate/
    • By jaro
      This module (github) does with site/assets/files what Ryan's DatabaseBackups module does with the database:
      Backup site/assets Download ZIP archive Upload ZIP archive Restore site/assets Motivation: This module can be the missing part for projects with content backup responsibility on the client's side: The client will be able to download DB and assets/files snapshots through the backend without filesystem access, thus backing up all content themselves.
      Release state alpha – do not use in production environments.
      Credits for the nice UI go to @ryan – I reused most of it and some other code from the DatabaseBackups module.
    • By NorbertH
      Is there a hook to do something right after cloning a page ?
      I want the page to be saved right after cloning it either from the button in the tree or from a lister, because saving the page triggers several calculations that are not triggered by just cloning the page.
       
      Thanks in advance !
    • By John W.
      Question 1
      I recently installed PW 3.0.62 for a new site and also have sites running older version of PW 3.x.
      Can I export the database on an older version of PW 3.x and import it to PW 3.0.62 without any issues?
       
      Question 2
      (This is kind of alternative to the above for long term use - and maybe a better solution...)
      On  the sites I've previously built I have templates (home, basic-page, contact) and fields that I commonly use, such as business_name, phone_1.  The last site I built is running PW 3.0.42.  I was considering cloning this into a local site and running the upgrade module to bring it up to PW 3.0.62. From there on out when I start I new project I could just run the PW upgrade module, copy the folder to the location for the new project and duplicate the database using the new projects name.

      So basically, I'll always keep a "blank slate" site that I can just run the PW upgrade on, then duplicate into a new project. This would cut down on the work and time spent having to re-create these common fields, that I use. From there, I would just add fields, templates, etc, specific for the new website project.

      Is this a sound approach to speed up development?

       
×
×
  • Create New...