Jump to content
flydev

Duplicator: Backup and move sites

Recommended Posts

Another issue report:

Right after installing and accessing Duplicator's Package Manager, when no package has ever been created:

2x PHP Warning: filesize(): stat failed for .../site/assets/logs/duplicator.txt in .../wire/core/FileLog.php:225
1× PHP Warning: filesize(): stat failed for .../site/assets/logs/duplicator.txt in .../wire/core/FileLog.php:237
1× PHP Warning: file(.../site/assets/logs/duplicator.txt): failed to open stream: No such file or directory in .../wire/core/FileLog.php:238

 

  • Thanks 1

Share this post


Link to post
Share on other sites
4 hours ago, flydev said:

@adrian I will test GoogleDrive, but if you choose to keep 10 backups, it should keep deleting the oldest backup and adding the new one without the need to use the "Remove Backup Packages Older Than" option. I remember and Its possible that I changed one settings last time I pushed an update. I will check.

Thanks for checking - it's definitely an issue here.

Share this post


Link to post
Share on other sites

Hi everyone! Does anyone know if it's possible to set the amazon S3 uploaded files to the infrequent access storage class?

Share this post


Link to post
Share on other sites

Hi @elabx

I did not know this property, I will make an option available for that.

 

I plan to add new features and test the module in the next 10 days, sorry I miss time 🔥

  • Like 5

Share this post


Link to post
Share on other sites

Good day, @flydev!

It's been a while science beta testing phase. Really glad the module is now released and gathering all the attention it deserves!

May I propose a little enhancement that seems reasonable to me and just may be of use to others?

I am using Duplicator to quickly create a local dev environment for a site (just not to practice cowboy coding straight on the server). When I do so the installer overrides the db settings in config file.

It seems like there is not an option to later copy a locally produced package back to production and replace the installation from the admin. One has to remove all the files (and probably drop all the tables in the existing database) and run the installation "from the clean slate". And one has to remember and fill the production db credentials.

Bit we could have an option to put the db credentials in the config-dev.php file instead (for local installation) and another one to remove config-dev.php from the installation if needed (for "back-to-production" installation).

P.S. While writing this I realized I do not really know what happens to the existing db tables if the db is not empty. Are the being dropped?

  • Like 3

Share this post


Link to post
Share on other sites

Hi @Ivan Gretsky :)

 

12 minutes ago, Ivan Gretsky said:

Bit we could have an option to put the db credentials in the config-dev.php file instead (for local installation) and another one to remove config-dev.php from the installation if needed (for "back-to-production" installation).

Sound good to me, look like I have some work to be done!

 

11 minutes ago, Ivan Gretsky said:

P.S. While writing this I realized I do not really know what happens to the existing db tables if the db is not empty. Are the being dropped?

Tables are simply dropped if they exist.

  • Like 4

Share this post


Link to post
Share on other sites

Yes it was planned to be done in first instance trough FTP protocol. Horst had give me some code to work with but since 2 months I am so busy with my works that I didn't found the time to try it.

But this feature is definitely on the top of the todo list.

  • Like 3

Share this post


Link to post
Share on other sites
49 minutes ago, flydev said:

Yes it was planned to be done in first instance trough FTP protocol. Horst had give me some code to work with but since 2 months I am so busy with my works that I didn't found the time to try it.

But this feature is definitely on the top of the todo list.

I just made a quick search and found this one - could be an easier replacement to ftp.

But could we make the 1st step even simpler - not care about the uploading side at all. Say we want to restore something already on the server, be it the package created on the server in the 1st place or uploaded via on-server file manager of ftp. And only provide the facility to kill the current installation and launch the new one from the backup?

  • Like 1

Share this post


Link to post
Share on other sites

Good day, @flydev!

I am trying to transfer my work (done locally on windows 10) to a production linux server. But after package got extracted with the installer, I got this instead of the file structure;
2018-02-21_15-33-09.png.4c1ff4c3b1f47e42075ea139dc472991.png

(filenames = what should be paths)

When manually extracting I get the same result. It seems like archiving does not work as expected on windows. Am I doing something wrong, forgetting something?

  • Like 1

Share this post


Link to post
Share on other sites
On 2018/2/21 at 9:01 PM, Ivan Gretsky said:

Good day, @flydev!

I am trying to transfer my work (done locally on windows 10) to a production linux server. But after package got extracted with the installer, I got this instead of the file structure;
2018-02-21_15-33-09.png.4c1ff4c3b1f47e42075ea139dc472991.png

(filenames = what should be paths)

When manually extracting I get the same result. It seems like archiving does not work as expected on windows. Am I doing something wrong, forgetting something?

I get the same result and the installation cannot proceed further. The package was built in localhost using Laragon. The destinated server is Ubuntu 17.10. I did the transfer from the same Ubuntu server to localhost for the same site before.

 

EDIT: I test the same package in the Windows Laragon localhost. It works nicely. The issue is happened in step 3. Does the backslashes that generated by Windows matter? Maybe DIRECTORY_SEPARATOR should or should not be used?

  • Like 2

Share this post


Link to post
Share on other sites

I found that the real cause is the php method realpath() used inside Duplicator.module.

Reference: http://php.net/manual/en/function.realpath.php#example-2719

The method is generating the backslashes in Windows, which makes the zip extraction look like this.

Here is my fix.

$source = realpath($source); //search this line
$source = str_replace('\\', '/', $source); //add this line

$file = realpath($pathname); //search this line
$file = str_replace('\\', '/', $file); //add this line

The installer runs nicely now.

  • Like 5

Share this post


Link to post
Share on other sites

Hello guys, just to let you know that I am working on the module.

 

@Karl_T  As i need the module working on windows ASAP, I tested your fix on my Windows 2008 r2 server / PHP 7.0.28 and it throw error on $zip->close(); with an error : "No Error" 😆

Which version of Windows and PHP you are working on please ?

  • Like 3

Share this post


Link to post
Share on other sites
24 minutes ago, flydev said:

Which version of Windows and PHP you are working on please ?

Windows 10 and php-7.1.6-Win32-V14-x64

I didn't try Windows to Windows yet.

EDIT: I just create a package and install it with no error.

  • Thanks 1

Share this post


Link to post
Share on other sites
14 minutes ago, Ivan Gretsky said:

Maybe we could use some 3rd party php zip library for this? Not to deal with all those cross-platform issues...

 

The problem of those php libs is that they have some dependencies and most of them require the deps installed trough composer.

Share this post


Link to post
Share on other sites
3 minutes ago, flydev said:

The problem of those php libs is that they have some dependencies and most of them require the deps installed trough composer.

https://php-download.com/

You may use this to get rid of composer, but I don't know how legit it is.

Share this post


Link to post
Share on other sites

I'm not aware of any crossplatform differences with the internal PHP ZIP functionality. What is the issue you are speaking from?

Also, I have (re)written a ZIP library in pure PHP that is only one single file. But this was in the times as the included ZIP in PHP wasn't available on many hosts.

  • Like 2

Share this post


Link to post
Share on other sites

I can bundle the vendor dir too but this solution do not enchant me.

 

@horst  check this post from Ivan :

 

I know that '\' is a char on Unix system and thus the extracted files are not parsed as a directory tree. also i know I have a working version somewhere, I just can't put my hand on it :(

 

  • Like 1

Share this post


Link to post
Share on other sites

My experiences in 15 years with PHP on windows told me to not make any (platform) differences with filesystem related stuff. Simply use the / forwardslash for everything as long as you can achieve all operations with PHP internal functions! (copy, delete, ZIP!, etc)

The one and only rule where you need to convert forwardslashes to backwardslashes on windows is, when you do calls in the commandline via exec() or system().

All other stuff is handled well from PHP internallly across all platforms. So, on windows you should not use functions like realpath() or dirname() without to convert backslashes to forwardslashes afterwards! Thats the only rule I followed all the time. (locally detect windows = convert \ to /)

  • Like 6

Share this post


Link to post
Share on other sites

I see, thanks for your input.

At this moment, Its working in the way : 

  • unix to unix ok
  • unix to windows ok
  • windows to unix, testing it right now
  • Like 3

Share this post


Link to post
Share on other sites

Hey @flydev - did you manage to look into the issue I had with Google Drive not adding more backups after the defined max limit was reached? Thank you!

  • Like 1

Share this post


Link to post
Share on other sites
On 3/16/2018 at 10:05 AM, flydev said:

Hello guys, just to let you know that I am working on the module.

Thanks! Hope you will have time for Dropbox too.

Share this post


Link to post
Share on other sites

Hey @flydev - I was debugging some Notices in my code and had Tracy in strict mode (via the Panel Selector) and came across this. Let me know if you need any help reproducing.

image.png.7bd9d062276629fa1bfc1af47f869517.png

  • Thanks 1

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Recently Browsing   0 members

    No registered users viewing this page.

  • Similar Content

    • By David Karich
      ProcessWire InputfieldRepeaterMatrixDuplicate
      Thanks to the great ProModule "RepeaterMatrix" I have the possibility to create complex repeater items. With it I have created a quite powerful page builder. Many different content modules, with many more possible design options. The RepeaterMatrix module supports the cloning of items, but only within the same page. Now I often have the case that very design-intensive pages and items are created. If you want to use a content module on a different page (e.g. in the same design), you have to rebuild each item manually every time.
      This module extends the commercial ProModule "RepeaterMatrix" by the function to duplicate repeater items from one page to another page. The condition is that the target field is the same matrix field from which the item is duplicated. This module is currently understood as proof of concept. There are a few limitations that need to be considered. The intention of the module is that this functionality is integrated into the core of RepeaterMatrix and does not require an extra module.
      Check out the screencast
      What the module can do
      Duplicate a repeater item from one page to another No matter how complex the item is Full support for file and image fields Multilingual support Support of Min and Max settings Live synchronization of clipboard between multiple browser tabs. Copy an item and simply switch the browser tab to the target page and you will immediately see the past button Support of multiple RepeaterMatrix fields on one page Configurable which roles and fields are excluded Duplicated items are automatically pasted to the end of the target field and set to hidden status so that changes are not directly published Automatic clipboard update when other items are picked Automatically removes old clipboard data if it is not pasted within 6 hours Delete clipboard itself by clicking the selected item again Benefit: unbelievably fast workflow and content replication What the module can't do
      Before an item can be duplicated in its current version, the source page must be saved. This means that if you make changes to an item and copy this, the old saved state will be duplicated Dynamic loading is currently not possible. Means no AJAX. When pasting, the target page is saved completely No support for nested repeater items. Currently only first level items can be duplicated. Means a repeater field in a repeater field cannot be duplicated. Workaround: simply duplicate the parent item Dynamic reloading and adding of repeater items cannot be registered. Several interfaces and events from the core are missing. The initialization occurs only once after the page load event Changelog
      1.0.4
      Bug fix: Various bug fixes and improvements in live synchronization Bug fix: Items are no longer inserted when the normal save button is clicked. Only when the past button is explicitly clicked Feature: Support of multiple repeater fields in one page Feature: Support of repeater Min/Max settings Feature: Configurable roles and fields Enhancement: Improved clipboard management Enhancement: Documentation improvement Enhancement: Corrected few typos #1 1.0.3
      Feature: Live synchronization Enhancement: Load the module only in the backend Enhancement: Documentation improvement 1.0.2
      Bug fix: Various bug fixes and improvements in JS functions Enhancement: Documentation improvement Enhancement: Corrected few typos 1.0.1
      Bug fix: Various bug fixes and improvements in the duplication process 1.0.0
      Initial release Support this module
      If this module is useful for you, I am very thankful for your small donation: Donate 5,- Euro (via PayPal – or an amount of your choice. Thank you!)
      Download this module
      > Github: https://github.com/FlipZoomMedia/InputfieldRepeaterMatrixDuplicate
      > PW module directory: https://modules.processwire.com/modules/inputfield-repeater-matrix-duplicate/
    • By jaro
      This module (github) does with site/assets/files what Ryan's DatabaseBackups module does with the database:
      Backup site/assets Download ZIP archive Upload ZIP archive Restore site/assets Motivation: This module can be the missing part for projects with content backup responsibility on the client's side: The client will be able to download DB and assets/files snapshots through the backend without filesystem access, thus backing up all content themselves.
      Release state alpha – do not use in production environments.
      Credits for the nice UI go to @ryan – I reused most of it and some other code from the DatabaseBackups module.
    • By NorbertH
      Is there a hook to do something right after cloning a page ?
      I want the page to be saved right after cloning it either from the button in the tree or from a lister, because saving the page triggers several calculations that are not triggered by just cloning the page.
       
      Thanks in advance !
    • By John W.
      Question 1
      I recently installed PW 3.0.62 for a new site and also have sites running older version of PW 3.x.
      Can I export the database on an older version of PW 3.x and import it to PW 3.0.62 without any issues?
       
      Question 2
      (This is kind of alternative to the above for long term use - and maybe a better solution...)
      On  the sites I've previously built I have templates (home, basic-page, contact) and fields that I commonly use, such as business_name, phone_1.  The last site I built is running PW 3.0.42.  I was considering cloning this into a local site and running the upgrade module to bring it up to PW 3.0.62. From there on out when I start I new project I could just run the PW upgrade module, copy the folder to the location for the new project and duplicate the database using the new projects name.

      So basically, I'll always keep a "blank slate" site that I can just run the PW upgrade on, then duplicate into a new project. This would cut down on the work and time spent having to re-create these common fields, that I use. From there, I would just add fields, templates, etc, specific for the new website project.

      Is this a sound approach to speed up development?

       
×
×
  • Create New...