flydev

Duplicator: Backup and move sites

Recommended Posts

Another issue report:

Right after installing and accessing Duplicator's Package Manager, when no package has ever been created:

2x PHP Warning: filesize(): stat failed for .../site/assets/logs/duplicator.txt in .../wire/core/FileLog.php:225
1× PHP Warning: filesize(): stat failed for .../site/assets/logs/duplicator.txt in .../wire/core/FileLog.php:237
1× PHP Warning: file(.../site/assets/logs/duplicator.txt): failed to open stream: No such file or directory in .../wire/core/FileLog.php:238

 

  • Thanks 1

Share this post


Link to post
Share on other sites
4 hours ago, flydev said:

@adrian I will test GoogleDrive, but if you choose to keep 10 backups, it should keep deleting the oldest backup and adding the new one without the need to use the "Remove Backup Packages Older Than" option. I remember and Its possible that I changed one settings last time I pushed an update. I will check.

Thanks for checking - it's definitely an issue here.

Share this post


Link to post
Share on other sites

Hi everyone! Does anyone know if it's possible to set the amazon S3 uploaded files to the infrequent access storage class?

Share this post


Link to post
Share on other sites

Hi @elabx

I did not know this property, I will make an option available for that.

 

I plan to add new features and test the module in the next 10 days, sorry I miss time 🔥

  • Like 5

Share this post


Link to post
Share on other sites

Good day, @flydev!

It's been a while science beta testing phase. Really glad the module is now released and gathering all the attention it deserves!

May I propose a little enhancement that seems reasonable to me and just may be of use to others?

I am using Duplicator to quickly create a local dev environment for a site (just not to practice cowboy coding straight on the server). When I do so the installer overrides the db settings in config file.

It seems like there is not an option to later copy a locally produced package back to production and replace the installation from the admin. One has to remove all the files (and probably drop all the tables in the existing database) and run the installation "from the clean slate". And one has to remember and fill the production db credentials.

Bit we could have an option to put the db credentials in the config-dev.php file instead (for local installation) and another one to remove config-dev.php from the installation if needed (for "back-to-production" installation).

P.S. While writing this I realized I do not really know what happens to the existing db tables if the db is not empty. Are the being dropped?

  • Like 3

Share this post


Link to post
Share on other sites

Hi @Ivan Gretsky :)

 

12 minutes ago, Ivan Gretsky said:

Bit we could have an option to put the db credentials in the config-dev.php file instead (for local installation) and another one to remove config-dev.php from the installation if needed (for "back-to-production" installation).

Sound good to me, look like I have some work to be done!

 

11 minutes ago, Ivan Gretsky said:

P.S. While writing this I realized I do not really know what happens to the existing db tables if the db is not empty. Are the being dropped?

Tables are simply dropped if they exist.

  • Like 4

Share this post


Link to post
Share on other sites

Yes it was planned to be done in first instance trough FTP protocol. Horst had give me some code to work with but since 2 months I am so busy with my works that I didn't found the time to try it.

But this feature is definitely on the top of the todo list.

  • Like 3

Share this post


Link to post
Share on other sites
49 minutes ago, flydev said:

Yes it was planned to be done in first instance trough FTP protocol. Horst had give me some code to work with but since 2 months I am so busy with my works that I didn't found the time to try it.

But this feature is definitely on the top of the todo list.

I just made a quick search and found this one - could be an easier replacement to ftp.

But could we make the 1st step even simpler - not care about the uploading side at all. Say we want to restore something already on the server, be it the package created on the server in the 1st place or uploaded via on-server file manager of ftp. And only provide the facility to kill the current installation and launch the new one from the backup?

  • Like 1

Share this post


Link to post
Share on other sites

Good day, @flydev!

I am trying to transfer my work (done locally on windows 10) to a production linux server. But after package got extracted with the installer, I got this instead of the file structure;
2018-02-21_15-33-09.png.4c1ff4c3b1f47e42075ea139dc472991.png

(filenames = what should be paths)

When manually extracting I get the same result. It seems like archiving does not work as expected on windows. Am I doing something wrong, forgetting something?

  • Like 1

Share this post


Link to post
Share on other sites
On 2018/2/21 at 9:01 PM, Ivan Gretsky said:

Good day, @flydev!

I am trying to transfer my work (done locally on windows 10) to a production linux server. But after package got extracted with the installer, I got this instead of the file structure;
2018-02-21_15-33-09.png.4c1ff4c3b1f47e42075ea139dc472991.png

(filenames = what should be paths)

When manually extracting I get the same result. It seems like archiving does not work as expected on windows. Am I doing something wrong, forgetting something?

I get the same result and the installation cannot proceed further. The package was built in localhost using Laragon. The destinated server is Ubuntu 17.10. I did the transfer from the same Ubuntu server to localhost for the same site before.

 

EDIT: I test the same package in the Windows Laragon localhost. It works nicely. The issue is happened in step 3. Does the backslashes that generated by Windows matter? Maybe DIRECTORY_SEPARATOR should or should not be used?

  • Like 2

Share this post


Link to post
Share on other sites

I found that the real cause is the php method realpath() used inside Duplicator.module.

Reference: http://php.net/manual/en/function.realpath.php#example-2719

The method is generating the backslashes in Windows, which makes the zip extraction look like this.

Here is my fix.

$source = realpath($source); //search this line
$source = str_replace('\\', '/', $source); //add this line

$file = realpath($pathname); //search this line
$file = str_replace('\\', '/', $file); //add this line

The installer runs nicely now.

  • Like 5

Share this post


Link to post
Share on other sites

Hello guys, just to let you know that I am working on the module.

 

@Karl_T  As i need the module working on windows ASAP, I tested your fix on my Windows 2008 r2 server / PHP 7.0.28 and it throw error on $zip->close(); with an error : "No Error" 😆

Which version of Windows and PHP you are working on please ?

  • Like 3

Share this post


Link to post
Share on other sites
24 minutes ago, flydev said:

Which version of Windows and PHP you are working on please ?

Windows 10 and php-7.1.6-Win32-V14-x64

I didn't try Windows to Windows yet.

EDIT: I just create a package and install it with no error.

  • Thanks 1

Share this post


Link to post
Share on other sites
14 minutes ago, Ivan Gretsky said:

Maybe we could use some 3rd party php zip library for this? Not to deal with all those cross-platform issues...

 

The problem of those php libs is that they have some dependencies and most of them require the deps installed trough composer.

Share this post


Link to post
Share on other sites
3 minutes ago, flydev said:

The problem of those php libs is that they have some dependencies and most of them require the deps installed trough composer.

https://php-download.com/

You may use this to get rid of composer, but I don't know how legit it is.

Share this post


Link to post
Share on other sites

I'm not aware of any crossplatform differences with the internal PHP ZIP functionality. What is the issue you are speaking from?

Also, I have (re)written a ZIP library in pure PHP that is only one single file. But this was in the times as the included ZIP in PHP wasn't available on many hosts.

  • Like 2

Share this post


Link to post
Share on other sites

I can bundle the vendor dir too but this solution do not enchant me.

 

@horst  check this post from Ivan :

 

I know that '\' is a char on Unix system and thus the extracted files are not parsed as a directory tree. also i know I have a working version somewhere, I just can't put my hand on it :(

 

  • Like 1

Share this post


Link to post
Share on other sites

My experiences in 15 years with PHP on windows told me to not make any (platform) differences with filesystem related stuff. Simply use the / forwardslash for everything as long as you can achieve all operations with PHP internal functions! (copy, delete, ZIP!, etc)

The one and only rule where you need to convert forwardslashes to backwardslashes on windows is, when you do calls in the commandline via exec() or system().

All other stuff is handled well from PHP internallly across all platforms. So, on windows you should not use functions like realpath() or dirname() without to convert backslashes to forwardslashes afterwards! Thats the only rule I followed all the time. (locally detect windows = convert \ to /)

  • Like 6

Share this post


Link to post
Share on other sites

I see, thanks for your input.

At this moment, Its working in the way : 

  • unix to unix ok
  • unix to windows ok
  • windows to unix, testing it right now
  • Like 3

Share this post


Link to post
Share on other sites

Hey @flydev - did you manage to look into the issue I had with Google Drive not adding more backups after the defined max limit was reached? Thank you!

  • Like 1

Share this post


Link to post
Share on other sites
On 3/16/2018 at 10:05 AM, flydev said:

Hello guys, just to let you know that I am working on the module.

Thanks! Hope you will have time for Dropbox too.

Share this post


Link to post
Share on other sites

Hey @flydev - I was debugging some Notices in my code and had Tracy in strict mode (via the Panel Selector) and came across this. Let me know if you need any help reproducing.

image.png.7bd9d062276629fa1bfc1af47f869517.png

  • Thanks 1

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now

  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By John W.
      Question 1
      I recently installed PW 3.0.62 for a new site and also have sites running older version of PW 3.x.
      Can I export the database on an older version of PW 3.x and import it to PW 3.0.62 without any issues?
       
      Question 2
      (This is kind of alternative to the above for long term use - and maybe a better solution...)
      On  the sites I've previously built I have templates (home, basic-page, contact) and fields that I commonly use, such as business_name, phone_1.  The last site I built is running PW 3.0.42.  I was considering cloning this into a local site and running the upgrade module to bring it up to PW 3.0.62. From there on out when I start I new project I could just run the PW upgrade module, copy the folder to the location for the new project and duplicate the database using the new projects name.

      So basically, I'll always keep a "blank slate" site that I can just run the PW upgrade on, then duplicate into a new project. This would cut down on the work and time spent having to re-create these common fields, that I use. From there, I would just add fields, templates, etc, specific for the new website project.

      Is this a sound approach to speed up development?

       
    • By John W.
      I just posted a question about moving PW from a resting domain to a production domain, after consideration I figure I would create a new post for a second question I have.
      I noticed the coming feature of client size image resizing. This will be really helpful, as I have some clients on a Satellite connection. I'm telling you, them trying to upload 15MB images, or them getting them straight on batch resizers (or Photoshop) to resize before upload is a real chore. Needless to say, Ryan has been doing an excellent job addressing this.
      My question, is that I have several clients on PW 2.x thru 3.x. Is there a really good guide on steps that I need to take to update the core to take advantage of the features of new releases? Are their perils to look out for?
      Yeah, after developing several sites, I feel a bit silly about asking this one, but, the docs doesn't seem to be clear in a "general" sense of upgrading the core across all versions.
      Thanks everyone for your time and assistance.
    • By suntrop
      Hi all
      I have big troubles with my DB backups and phpMyAdmin. I can't manage to import a backup.
      The phpMyAdmin export settings are left to the defaults. The error message I get:
      #1064 - You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'INSERT INTO `caches` (`name`, `data`, `expires`) VALUES ('ModulesUninstalled.in' at line 11 When I delete the complete 'cache' section there are almost for all tables the same errors. (part of the dump attached)
      I had this problem before and back then (couple months ago) I think I copied almost every table individually. I thought it was specific to that DB. But I can't do a dump/import of any of my PW installations.
      MySQL: 5.6.27
      PHP version: 7.0.11
      phpMyAdmin: 4.5.5.1
      Anybody knows that problem? Or what to do?
      phpMyAdmin SQL Dump.sql
    • By vwatson
      Is it possible to add check boxes in order to select multiple backup files and delete them all at once, rather than having to click each one, then Delete?
      Also it would be helpful to make the Restore process more distinct from Deleting.  I must have inadvertently clicked the wrong button once and restored a backup unintentionally, wiping out a day's work. My mistake of course. I obviously wasn't paying close enough attention.  
    • By Fantomas
      Hi all,
      I'm still new to processwire and now I have to create the first hook.
      I need to modify the page-clone module, so there is a field in pages that has to be empty after cloning the page. How could I start this task?