Jump to content
flydev

Duplicator: Backup and move sites

Recommended Posts

I notice on the Module settings that the username and password are displayed in plain text.

I've a few admins that I wouldn't like to be able to view these. Is masking these on the roadmap or even encrypting them?

Does this also mean that they're stored in the database in plain text?

Thanks

  • Like 1

Share this post


Link to post
Share on other sites

Hi 

One more question

I have 2 FTP applications here successfully connecting to a destination server over FTPS SSL/TLS 

 

When I add the exact same credentials to Duplicator, I get the following

Quote

2018-09-06 15:00:43:  - job finished in 110.8802sec
2018-09-06 15:00:43:  [FTP] error: cannot upload file <2018-09-06_14-58-52-www.domain.co.uk.package.zip> on server.
2018-09-06 14:59:40:  FTP: starting upload of /home/sites/domain.co.uk/public_html/site/assets/backups/2018-09-06_14-58-52-www.domain.co.uk.package.zip
2018-09-06 14:59:40:  FTP: directory listing of Backups successfull.
2018-09-06 14:59:40:  FTP: retrieving directory listing of Backups...
2018-09-06 14:59:40:  FTP: directory listing of Sharey01 successfull.
2018-09-06 14:59:40:  FTP: retrieving directory listing of Sharey01...
2018-09-06 14:59:39:  FTP: logged in.
2018-09-06 14:59:39:  FTP: connection established.
2018-09-06 14:59:39:  FTP: connecting to myserver.com:21...
2018-09-06 14:59:39:  - package built successfully in 46.558444sec

So it looks like Duplicator is successfully building the package, logging in to the server and listing the directory files. It only fails to actually upload the ZIP

There are no size restrictions on the server and there are I have disconnected the other FTP apps to ensure that's not the issue either.

Any ideas why the upload might fail? This only happens on TLS/SSL and not under 'regular' unencrypted FTP

Thanks

 

  • Like 1

Share this post


Link to post
Share on other sites

Found a problem when the webiste is on a different port number (http:\\127.0.0.1:81) the backup fails. The logs list:

1 hour ago
2018-09-07 12:53:33
admin /processwire/setup/duplicator/?action=backup_n… - package build failed.
1 hour ago
2018-09-07 12:53:33
admin /processwire/setup/duplicator/?action=backup_n… - an error occured during package build.
1 hour ago
2018-09-07 12:53:33
admin /processwire/setup/duplicator/?action=backup_n… - an error occured while building the ProcessWire structure.

 

It is adding the colon(:) into the filename of the zips. I changed the code in DupUtil.php on line 48 to filter the colon(:) out and replace it with a period:

FROM:    $filename = $date . '-' . str_replace(' ', '_', $name); 

TO:         $filename = $date . '-' . str_replace(':', '.', str_replace(' ', '_', $name));

There may be a better place to put this in but it does the trick!

Thanks for the plugin - it's my goto for securing backups.

Yours truly

Robert

  • Like 1

Share this post


Link to post
Share on other sites

@flydev, just trying this module for the first time today. Very nice! The subfolder exclusion options are coming in handy.

Do you think the main Duplicator.module should automatically install ProcessDuplicator.module? Otherwise if you install by entering the Duplicator classname then the Process module doesn't get installed and it's not immediately obvious why the link from the config screen to the Package Manager results in an "Unrecognized path" error.

  • Like 1
  • Thanks 1

Share this post


Link to post
Share on other sites

@Robin S Thanks for trying it, glad you like it !  

The Process module is not really required because Duplicator could be, if needed, only run from CRON task. The link to he process module should had been removed if the process module was not present, just checking my todo list, it was into. 

 

A known issue when restoring a package :

On 6/21/2018 at 11:04 AM, flydev said:

On the current state of the module, it is usable for small website, I mean small database (I am aware of the issue which include the  parent directory in the compressed package which make the package non deployable by the installer).

So for restoring it, we need to manually move the site files out of the container dir.

 

About AmazonS3, @Rudy asked the ability to upload package on sub-bucket (I forgot this one, sorry Rudy!) : https://github.com/flydev-fr/Duplicator/issues/1

 

I will try to code theses corrections soon so the V1 will work as expected, but I really want to begin to work on the Duplicator V2 implementing encryption and handling gigabits sized databases. Want To Buy time or vacation 🔥

 

 

  • Like 5

Share this post


Link to post
Share on other sites
11 hours ago, flydev said:

The Process module is not really required because Duplicator could be, if needed, only run from CRON task.

I see what you mean, although there isn't really any "cost" to a Process module being installed because it doesn't do anything unless you visit its page. So for the sake of new users trying to understand the different ways they can use Duplicator I still think it would be good to auto-install the Process module and CRON-only users can just ignore it, or uninstall it if they don't want to see the item in the Setup menu for some reason. But now that I have used Duplicator for a bit I could figure it out either way.

  • Like 2

Share this post


Link to post
Share on other sites

Is there a chance to exclude files by regex?

I'm thinking about something like "\.\d+x\d+\." to exclude any image variant created from the API, for example.

  • Like 1

Share this post


Link to post
Share on other sites
On 4/19/2018 at 9:03 AM, Karl_T said:

I have encountered an issue for Duplicator ver 1.2.9.

 I am moving a site from localhost(windows 10) to Amazon Linux. I use RDS, which is a standalone server for database, so the dbHost is not localhost.

The issue happened in the installation process after entering the database data and proceed. There's an error telling config file cannot be found. Then I go to config.php, there are at least 4 places are using old information. They are dbHost, dbName, dbUser, dbPass and httpHosts. I think for some reasons the config.php did not update at all after the installation complete. After I manually update the fields, the site works fine now. This means database import is working fine. I wonder if it is the dbHost matters. It is a long URL ended with xxx.rds.amazonaws.com.

I had the same problem a couple of times now. For example when I try to install a backup of a live site on my localhost (Running on Mamp) or an a web server I always get the error config file cannot be found.
The issue happened in the installation process after entering the database data and proceed.
Maybe this is happening when the website is in another subdirectory and not the web root? 
The duplicator always makes a subdirectory when unzipping the files and then looks for the config in the root folder.
After I manually change the config file, the site was showing up and I could see all the content in the admin, but had some strange errors  on the frontend.

  • Like 1

Share this post


Link to post
Share on other sites

I remember falling into the same trap. I didn't modify the generated config.php but simply moved the contents from the unzipped subdirectory one up before I submit the database information.

  • Like 2

Share this post


Link to post
Share on other sites
49 minutes ago, Autofahrn said:

I remember falling into the same trap. I didn't modify the generated config.php but simply moved the contents from the unzipped subdirectory one up before I submit the database information.

The installation worked that way. But Iam still getting errors on the frontend saying Warning: preg_replace(): JIT compilation failed: no more memory
Any Ideas, what is causing this? When I use the site profile exporter and install the profile everything is working fine.

  • Like 1

Share this post


Link to post
Share on other sites

I changed the PHP version to 5.6 and these errors are gone now.
However my markup regions seem not to be working. But when logged into the admin everything on the frontend works fine.
When logged out I get an error saying "no regions" and the site is messed up.
Iam about to give up on this and just use the site profile exporter for now 😞

 

  • Like 1

Share this post


Link to post
Share on other sites
3 hours ago, jploch said:

The duplicator always makes a subdirectory when unzipping the files and then looks for the config in the root folder.

Sorry dude, I am aware of this issue but forgot to update the first post and the github doc. So yes, basically, In the last version of Duplicator, its necessary (but unwanted) to move the unzipped data to the parent dir..

And about your last issue, try what its said in the github issue linked by @netcarver and more precisely this one : https://github.com/composer/composer/issues/7836#issuecomment-452341683

And just saying, I have not tested the module with PHP 7.x and it might have some issues on this version.

 

2 hours ago, jploch said:

Iam about to give up on this and just use the site profile exporter for now 😞

sorry for the inconveniences, I hope to get the time to finally work on the V2 🤞 but I promise you guys to take a look at the actual version and make a revision in the last 10 days or ask to moderators to ban me from the forum ! 🤐 😅

 

  • Like 2

Share this post


Link to post
Share on other sites
15 minutes ago, flydev said:

And just saying, I have not tested the module with PHP 7.x and it might have some issues on this version.

Thats fine with me. If I use Mamp with PHP 5.6 this issue is solved.

15 minutes ago, flydev said:

sorry for the inconveniences, I hope to get the time to finally work on the V2 🤞 but I promise you guys to take a look at the actual version and make a revision in the last 10 days or ask to moderators to ban me from the forum ! 🤐 😅

No worries! Keep up the good work! Looking forward to use this one day 🙂
I still have no clue what stops the markup regions from working though.
 

  • Thanks 1

Share this post


Link to post
Share on other sites

A small news about Duplicator.

There is a work in progress on a development version (v1.3.10-ATO) - nothing which can harm your server and could be already tested on different setup. You can download this dev version on GitHub :  https://github.com/flydev-fr/Duplicator/archive/dev.zip or at the end of this post.

The module still contain minor issues but the overall processing is faster than before and packages can be built and restored as well. Thanks to @Autofahrn for putting his hand in the module! Autofahrn also added a great feature which give the possibility to exclude file through regular expression. Also, image variations created by ProcessWire are now excluded by default.

 

 

 

Duplicator-dev-1.3.10-ATO.zip

  • Like 5

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By NorbertH
      Is there a hook to do something right after cloning a page ?
      I want the page to be saved right after cloning it either from the button in the tree or from a lister, because saving the page triggers several calculations that are not triggered by just cloning the page.
       
      Thanks in advance !
    • By David Karich
      Thanks to the great Pro module "RepeaterMatrix" I have the possibility to create complex repeater items. With it I have created a quite powerful page builder. Many different content modules, with many more possible design options. The RepeaterMatrix module supports the cloning of items, but only within the same page. Now I often have the case that very design-intensive pages and items are created. If you want to use this module on a different page (e.g. in the same design), you have to rebuild each item manually every time.
      With this proof of concept I have created a module which adds the feature to copy a repeater item to the clipboard so that you can paste this item to another page with the same repeater field. The module has been developed very rudimentarily so far. It is currently not possible to copy nested items. There is also no check of Min/Max. You can also only copy items that have the same field on different pages. And surely you can solve all this more elegantly with AJAX. But personally I lack the deeper understanding of the repeaters. Also missing on the Javascript side are event triggers for the repeaters, which would make it easier. Like e.g. RepeaterItemInitReady or similar.
      it would be great if @ryan would implement this functionality in the core of RepeaterMatrix. I think he has better ways to implement this. Or what do you think, Ryan?
      Everybody is welcome to work on this module and improve it, if it should not be integrated into the matrix core. Therefore I put it for testing and as download on GitHub: https://github.com/FlipZoomMedia/InputfieldRepeaterMatrixDublicate
      You can best see the functionality in the screencast: 
       
    • By John W.
      Question 1
      I recently installed PW 3.0.62 for a new site and also have sites running older version of PW 3.x.
      Can I export the database on an older version of PW 3.x and import it to PW 3.0.62 without any issues?
       
      Question 2
      (This is kind of alternative to the above for long term use - and maybe a better solution...)
      On  the sites I've previously built I have templates (home, basic-page, contact) and fields that I commonly use, such as business_name, phone_1.  The last site I built is running PW 3.0.42.  I was considering cloning this into a local site and running the upgrade module to bring it up to PW 3.0.62. From there on out when I start I new project I could just run the PW upgrade module, copy the folder to the location for the new project and duplicate the database using the new projects name.

      So basically, I'll always keep a "blank slate" site that I can just run the PW upgrade on, then duplicate into a new project. This would cut down on the work and time spent having to re-create these common fields, that I use. From there, I would just add fields, templates, etc, specific for the new website project.

      Is this a sound approach to speed up development?

       
    • By John W.
      I just posted a question about moving PW from a resting domain to a production domain, after consideration I figure I would create a new post for a second question I have.
      I noticed the coming feature of client size image resizing. This will be really helpful, as I have some clients on a Satellite connection. I'm telling you, them trying to upload 15MB images, or them getting them straight on batch resizers (or Photoshop) to resize before upload is a real chore. Needless to say, Ryan has been doing an excellent job addressing this.
      My question, is that I have several clients on PW 2.x thru 3.x. Is there a really good guide on steps that I need to take to update the core to take advantage of the features of new releases? Are their perils to look out for?
      Yeah, after developing several sites, I feel a bit silly about asking this one, but, the docs doesn't seem to be clear in a "general" sense of upgrading the core across all versions.
      Thanks everyone for your time and assistance.
    • By suntrop
      Hi all
      I have big troubles with my DB backups and phpMyAdmin. I can't manage to import a backup.
      The phpMyAdmin export settings are left to the defaults. The error message I get:
      #1064 - You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'INSERT INTO `caches` (`name`, `data`, `expires`) VALUES ('ModulesUninstalled.in' at line 11 When I delete the complete 'cache' section there are almost for all tables the same errors. (part of the dump attached)
      I had this problem before and back then (couple months ago) I think I copied almost every table individually. I thought it was specific to that DB. But I can't do a dump/import of any of my PW installations.
      MySQL: 5.6.27
      PHP version: 7.0.11
      phpMyAdmin: 4.5.5.1
      Anybody knows that problem? Or what to do?
      phpMyAdmin SQL Dump.sql
×
×
  • Create New...