☁️ Duplicator: Backup and move sites

[szabesz]

Another issue report:

Right after installing and accessing Duplicator's Package Manager, when no package has ever been created:

2x PHP Warning: filesize(): stat failed for .../site/assets/logs/duplicator.txt in .../wire/core/FileLog.php:225
1× PHP Warning: filesize(): stat failed for .../site/assets/logs/duplicator.txt in .../wire/core/FileLog.php:237
1× PHP Warning: file(.../site/assets/logs/duplicator.txt): failed to open stream: No such file or directory in .../wire/core/FileLog.php:238

 

[adrian]

4 hours ago, flydev said:

@adrian I will test GoogleDrive, but if you choose to keep 10 backups, it should keep deleting the oldest backup and adding the new one without the need to use the "Remove Backup Packages Older Than" option. I remember and Its possible that I changed one settings last time I pushed an update. I will check.

Thanks for checking - it's definitely an issue here.

[elabx]

Hi everyone! Does anyone know if it's possible to set the amazon S3 uploaded files to the infrequent access storage class?

[flydev]

Hi @elabx

I did not know this property, I will make an option available for that.

 

I plan to add new features and test the module in the next 10 days, sorry I miss time ?

[Ivan Gretsky]

Good day, @flydev!

It's been a while science beta testing phase. Really glad the module is now released and gathering all the attention it deserves!

May I propose a little enhancement that seems reasonable to me and just may be of use to others?

I am using Duplicator to quickly create a local dev environment for a site (just not to practice cowboy coding straight on the server). When I do so the installer overrides the db settings in config file.

It seems like there is not an option to later copy a locally produced package back to production and replace the installation from the admin. One has to remove all the files (and probably drop all the tables in the existing database) and run the installation "from the clean slate". And one has to remember and fill the production db credentials.

Bit we could have an option to put the db credentials in the config-dev.php file instead (for local installation) and another one to remove config-dev.php from the installation if needed (for "back-to-production" installation).

P.S. While writing this I realized I do not really know what happens to the existing db tables if the db is not empty. Are the being dropped?

[flydev]

Hi @Ivan Gretsky

 

12 minutes ago, Ivan Gretsky said:

Bit we could have an option to put the db credentials in the config-dev.php file instead (for local installation) and another one to remove config-dev.php from the installation if needed (for "back-to-production" installation).

Sound good to me, look like I have some work to be done!

 

11 minutes ago, Ivan Gretsky said:

P.S. While writing this I realized I do not really know what happens to the existing db tables if the db is not empty. Are the being dropped?

Tables are simply dropped if they exist.

[Ivan Gretsky]

Do you see any chance to restore a site from the package from admin, @flydev?

[flydev]

Yes it was planned to be done in first instance trough FTP protocol. Horst had give me some code to work with but since 2 months I am so busy with my works that I didn't found the time to try it.

But this feature is definitely on the top of the todo list.

[Ivan Gretsky]

49 minutes ago, flydev said:

Yes it was planned to be done in first instance trough FTP protocol. Horst had give me some code to work with but since 2 months I am so busy with my works that I didn't found the time to try it.

But this feature is definitely on the top of the todo list.

I just made a quick search and found this one - could be an easier replacement to ftp.

But could we make the 1st step even simpler - not care about the uploading side at all. Say we want to restore something already on the server, be it the package created on the server in the 1st place or uploaded via on-server file manager of ftp. And only provide the facility to kill the current installation and launch the new one from the backup?

[Ivan Gretsky]

Good day, @flydev!

I am trying to transfer my work (done locally on windows 10) to a production linux server. But after package got extracted with the installer, I got this instead of the file structure;

(filenames = what should be paths)

When manually extracting I get the same result. It seems like archiving does not work as expected on windows. Am I doing something wrong, forgetting something?

[Karl_T]

On 2018/2/21 at 9:01 PM, Ivan Gretsky said:

Good day, @flydev!

I am trying to transfer my work (done locally on windows 10) to a production linux server. But after package got extracted with the installer, I got this instead of the file structure;

(filenames = what should be paths)

When manually extracting I get the same result. It seems like archiving does not work as expected on windows. Am I doing something wrong, forgetting something?

I get the same result and the installation cannot proceed further. The package was built in localhost using Laragon. The destinated server is Ubuntu 17.10. I did the transfer from the same Ubuntu server to localhost for the same site before.

 

EDIT: I test the same package in the Windows Laragon localhost. It works nicely. The issue is happened in step 3. Does the backslashes that generated by Windows matter? Maybe DIRECTORY_SEPARATOR should or should not be used?

[Karl_T]

I found that the real cause is the php method realpath() used inside Duplicator.module.

Reference: http://php.net/manual/en/function.realpath.php#example-2719

The method is generating the backslashes in Windows, which makes the zip extraction look like this.

Here is my fix.

$source = realpath($source); //search this line
$source = str_replace('\\', '/', $source); //add this line

$file = realpath($pathname); //search this line
$file = str_replace('\\', '/', $file); //add this line

The installer runs nicely now.

[flydev]

Hello guys, just to let you know that I am working on the module.

 

@Karl_T  As i need the module working on windows ASAP, I tested your fix on my Windows 2008 r2 server / PHP 7.0.28 and it throw error on $zip->close(); with an error : "No Error" ?

Which version of Windows and PHP you are working on please ?

[Ivan Gretsky]

Maybe we could use some 3rd party php zip library for this? Not to deal with all those cross-platform issues...

 

[Karl_T]

24 minutes ago, flydev said:

Which version of Windows and PHP you are working on please ?

Windows 10 and php-7.1.6-Win32-V14-x64

I didn't try Windows to Windows yet.

EDIT: I just create a package and install it with no error.

[flydev]

14 minutes ago, Ivan Gretsky said:

Maybe we could use some 3rd party php zip library for this? Not to deal with all those cross-platform issues...

 

The problem of those php libs is that they have some dependencies and most of them require the deps installed trough composer.

[Karl_T]

3 minutes ago, flydev said:

The problem of those php libs is that they have some dependencies and most of them require the deps installed trough composer.

https://php-download.com/

You may use this to get rid of composer, but I don't know how legit it is.

[horst]

I'm not aware of any crossplatform differences with the internal PHP ZIP functionality. What is the issue you are speaking from?

Also, I have (re)written a ZIP library in pure PHP that is only one single file. But this was in the times as the included ZIP in PHP wasn't available on many hosts.

[flydev]

I can bundle the vendor dir too but this solution do not enchant me.

 

@horst  check this post from Ivan :

 

I know that '\' is a char on Unix system and thus the extracted files are not parsed as a directory tree. also i know I have a working version somewhere, I just can't put my hand on it

 

[horst]

My experiences in 15 years with PHP on windows told me to not make any (platform) differences with filesystem related stuff. Simply use the / forwardslash for everything as long as you can achieve all operations with PHP internal functions! (copy, delete, ZIP!, etc)

The one and only rule where you need to convert forwardslashes to backwardslashes on windows is, when you do calls in the commandline via exec() or system().

All other stuff is handled well from PHP internallly across all platforms. So, on windows you should not use functions like realpath() or dirname() without to convert backslashes to forwardslashes afterwards! Thats the only rule I followed all the time. (locally detect windows = convert \ to /)

[flydev]

I see, thanks for your input.

At this moment, Its working in the way : 

• unix to unix ok
• unix to windows ok
• windows to unix, testing it right now

[adrian]

Hey @flydev - did you manage to look into the issue I had with Google Drive not adding more backups after the defined max limit was reached? Thank you!

[flydev]

Working on it @adrian

[szabesz]

On 3/16/2018 at 10:05 AM, flydev said:

Hello guys, just to let you know that I am working on the module.

Thanks! Hope you will have time for Dropbox too.

[adrian]

Hey @flydev - I was debugging some Notices in my code and had Tracy in strict mode (via the Panel Selector) and came across this. Let me know if you need any help reproducing.