Jump to content

☁️ Duplicator: Backup and move sites


flydev

Recommended Posts

I have Duplicator running smoothly on several installs. Today I felt I needed to upgrade to the dev version (sometimes you feel crazy ?), but I'm running into this line:

if($zip->open($zipfile) !== true) throw new WireException("Unable to re-open ZIP: $zipfile");

I'm not sure how to debug this. I'm running PHP 7.2.17. ZipArchive has been installed. The Duplicator log also doesn't show any errors. Any pointers how to debug this?

  • Like 1
Link to comment
Share on other sites

18 minutes ago, arjen said:

The Duplicator log also doesn't show any errors. Any pointers how to debug this?

Did you check the additional logfile written into the backup directory? That should at least show "CLOSING ZIP" and "OPENING ZIP" along with the filename.

You may check, if that file actually exists on your server. If it does exist and the re-open fails its probably best to modify the code to get the error, something like:

	$res = $zip->open($zipfile);
	if($res !== true) throw new WireException("Unable to re-open ZIP ({$zipfile}): {$res}");

https://www.php.net/manual/en/ziparchive.open.php#117339

Anyway, you may set the flush margin (DUP_ZIP_FLUSH_MBYTES in Duplicator.module) to a higher value to disable this close&re-open mechanism.

  • Like 2
Link to comment
Share on other sites

Thanks for getting back on this.

The file doesn't exist on the server. I currently am testing on another install (same server same php version) and here it's working fine (as in it creates the zip flawless), but the logs in the ProcessModule says:

2019-05-12 13:06:32:  - package build failed.
2019-05-12 13:06:32:  - package build failed,  doesn't exist

When I compare the verbose logs between the two installs I notice this:

Install which fails tries to open after closing:

Quote

0.0s: CLOSING ZIP: /**/public/site/assets/backups/2019-05-11_13-52-12.package2.zip
1.4s: OPENING ZIP: /**/public/site/assets/backups/2019-05-11_13-52-12.package2.zip
Done in 1.41s

Install which succeeds only closes:

Quote

12.4s: CLOSING ZIP: /**/public/site/assets/backups/2019-05-12_13-06-08.package2.zip
Done in 22.43s

I will investigate further.

Link to comment
Share on other sites

The second install is probably working on a smaller site (less than 200 MBytes of data), so the re-opening mechanism is not triggered at all.

I guess you edited the pathnames, since the /**/ looks weird to me and I'm missing the domain name.

1 hour ago, arjen said:

2019-05-12 13:06:32:  - package build failed.
2019-05-12 13:06:32:  - package build failed,  doesn't exist 

Does your domain name contains a dash? Then this fix should help:

 

  • Like 2
Link to comment
Share on other sites

hello,

maybe the topic has already been discussed, but unfortunately I didn't find anything to it.
i have the problem that my site has to run under windows and iis, but Duplicator doesn't create a complete archive. the database can be archived, that seems to be complete, but the files don't. the archive always has only 1 KB.

does anyone have an idea?


 

Link to comment
Share on other sites

18 hours ago, entschleunigung said:

hello,

maybe the topic has already been discussed, but unfortunately I didn't find anything to it.
i have the problem that my site has to run under windows and iis, but Duplicator doesn't create a complete archive. the database can be archived, that seems to be complete, but the files don't. the archive always has only 1 KB.

does anyone have an idea?


 

I ran Duplicator on a Windows Server 2012, IIS and PHP 7.2.18 without issue, which version of Duplicator have you tested ?

Link to comment
Share on other sites

On 5/12/2019 at 2:50 PM, Autofahrn said:

Does your domain name contains a dash?

No, the domain name doesn't have a dash, but the file path does. These look like: /homes/username-tld/sites/domain.tld/. Perhaps the dash in the file path is the problem? I'll check out your fix later on. Thanks!

Link to comment
Share on other sites

19 minutes ago, arjen said:

Perhaps the dash in the file path is the problem?

Don't think so. The fix only retrieves the timestamp from filename differently, it does not look at the path.

Edit: just saw:

Quote

package build failed,  doesn't exist

There normally should be a filename between "failed," and "doesn't". Did you specify something in the name field?

Link to comment
Share on other sites

1 hour ago, flydev said:

I ran Duplicator on a Windows Server 2012, IIS and PHP 7.2.18 without issue, which version of Duplicator have you tested ?

hi fly,

I use the following environment:

Windows Server 2016
Version     1.2.9 
PHP Version 7.2.7

Duplicator Log:

13.05.2019 14:29:20:  - job finished in 1.102115sec
13.05.2019 14:29:20:  - package saved in local folder: 2019-05-13_14-29-19-xyz.package.zip
13.05.2019 14:29:20:  - package built successfully in 1.097146sec

as I said before, the database archive works perfectly, the rest unfortunately doesn't.

 

thank you

Link to comment
Share on other sites

1 hour ago, entschleunigung said:

Version     1.2.9 

Please upgrade to the dev version 1.3.10-ATO here :  https://github.com/flydev-fr/Duplicator/tree/dev

 

Steps :

- Rename the Duplicator folder to .Duplicator

- Download the dev version and extract the folder in the modules directory

- Rename Duplicator-dev to Duplicator

- Go to Modules > Refresh

 

It should works.

  • Like 2
  • Thanks 1
Link to comment
Share on other sites

48 minutes ago, flydev said:

It should works

thank you fly, 

thank you. now it works, but only with one exception. the package manager doesn't show the backups on the overview page, but you can find them under /site/assets/backups. i also tested different browsers, always with the same result. i also deleted the compiled files under module, no changes. 

and the logs:

14.05.2019 12:55:29:  - package build failed.
14.05.2019 12:55:29:  - package build failed,  doesn't exist

no stress, for me this is a small beauty error, i can live with it as long as the backups are made. but i don't mind if the overview page would work ?

thx

 

 

  • Like 1
Link to comment
Share on other sites

On 4/14/2019 at 8:43 PM, Autofahrn said:

Btw, @flydev, do you plan to include the suggested domain name fix into ProcessDuplicator::getPackagesDetails to support dashes in domain names?

Done. pushed the update in Duplicator 1.3.11-ATO.

 

5 hours ago, entschleunigung said:

but only with one exception. the package manager doesn't show the backups on the overview page

Does your domain name contain a dash char ?  If yes, try the latest update (v1.3.11-ATO).

  • Like 2
  • Thanks 1
Link to comment
Share on other sites

I've updated to the latest dev and unfortunately it is not working. 

2019-05-14 20:39:11:  - package build failed.
2019-05-14 20:39:11:  Unable to re-open ZIP: /home/xxxxxx-nl/webapps/xxxxxx-nl/public/site/assets/backups/2019-05-14_20-39-09-xxxxxx.nl.package2.zip

I tried upping DUP_ZIP_FLUSH_MBYTES and played with other settings, but no luck. I'm unsure why. For now I'll keep running the previous version. Thanks for the hard work on this module.

  • Like 2
Link to comment
Share on other sites

16 hours ago, flydev said:

Does your domain name contain a dash char ?  If yes, try the latest update (v1.3.11-ATO).

yes, that's the case with me. i have now installed the last update and everything works fine. existing backups now appear in the overview.

thank you
 

  • Like 2
Link to comment
Share on other sites

14 hours ago, arjen said:

I tried upping DUP_ZIP_FLUSH_MBYTES and played with other settings, but no luck. I'm unsure why.

Well, I guess your site contains a lot of data so the re-opening still is triggered. I'm really interested in the returned error number, did you try to add the suggested code change?

Otherwise you may simply disable the whole block in duplicator.module:

		if(false
		&& ($fragmentBytes >= self::DUP_ZIP_FLUSH_MBYTES*1024*1024))
			{
			$zipLog->verbose("CLOSING ZIP: {$zipfile}");
			$zip->close();
			$zipLog->verbose("OPENING ZIP: {$zipfile}");
//			if($zip->open($zipfile) !== true) throw new WireException("Unable to re-open ZIP: $zipfile");
			$res = $zip->open($zipfile);
			if($res !== true) throw new WireException("Unable to re-open ZIP ({$zipfile}): {$res}");
			$zipLog->verbose("OPENED ZIP: {$zipfile}");
			$fragmentBytes = 0;
			set_time_limit(300);
			}

@flydev, maybe we should have DUP_ZIP_FLUSH_MBYTES configurable with that option disabled when not set.

  • Like 2
Link to comment
Share on other sites

5 hours ago, Autofahrn said:

@flydev, maybe we should have DUP_ZIP_FLUSH_MBYTES configurable with that option disabled when not set.

Just pushed an update - v1.3.12-ATO - the quantity of megabytes saved before an archive flush is now configurable. When the value is empty or equal to 0 then the option is disabled.

 

5 hours ago, Autofahrn said:

if(false

@Autofahrn what is this check ? I don't have it in my code ?

Edited by flydev
link to the dev branch
  • Like 2
Link to comment
Share on other sites

7 hours ago, Autofahrn said:

Well, I guess your site contains a lot of data so the re-opening still is triggered. I'm really interested in the returned error number, did you try to add the suggested code change?

Sorry mate, missed this one. It is error 9.

2019-05-15 19:00:26: Unable to re-open ZIP (/home/xxxxxx-nl/webapps/xxxxxx-nl/public/site/assets/backups/2019-05-15_19-00-24-xxxxxx.nl.package2.zip): 9

When I disable that code block I get:

2019-05-15 19:04:00:  - package build failed.
2019-05-15 19:04:00:  - an error occured during package build.
2019-05-15 19:04:00:  - an error occured while building the ProcessWire structure: /home/xxxxxx-nl/webapps/xxxxxx-nl/public/site/assets/backups/2019-05-15_19-03-56-xxxxxx.nl.package2.zip root: /home/xxxxxx-nl/webapps/xxxxxx-nl/public/ Array (     [filename] => 2019-05-15_19-03-56-xxxxxx.nl.package2.zip     [folder] => /home/xxxxxx-nl/webapps/xxxxxx-nl/public/site/assets/backups/     [exclude] => Array         (             [0] => /site/assets/cache/WireTempDir             [1] => /site/assets/backups             [2] => /site/assets/cache             [3] => /site/assets/sessions             [4] => %\.\d+x\d+\.[^/]+$%             [5] =>          )      [extension] => Array         (         )      [allowHidden] => 1 )
2019-05-15 19:03:56:  - package build failed.
2019-05-15 19:03:56:  - an error occured during package build.
2019-05-15 19:03:56:  - an error occured while building the ProcessWire structure: /home/xxxxxx-nl/webapps/xxxxxx-nl/public/site/assets/backups/2019-05-15_19-03-52-xxxxxx.nl.package2.zip root: /home/xxxxxx-nl/webapps/xxxxxx-nl/public/ Array (     [filename] => 2019-05-15_19-03-52-xxxxxx.nl.package2.zip     [folder] => /home/xxxxxx-nl/webapps/xxxxxx-nl/public/site/assets/backups/     [exclude] => Array         (             [0] => /site/assets/cache/WireTempDir             [1] => /site/assets/backups             [2] => /site/assets/cache             [3] => /site/assets/sessions             [4] => %\.\d+x\d+\.[^/]+$%             [5] =>          )      [extension] => Array         (         )      [allowHidden] => 1 )

Thanks again! I currently use 1.3.11.

  • Like 3
Link to comment
Share on other sites

4 minutes ago, arjen said:

It is error 9.

ok, "no such file". No surprise, that re-opening and the package build fail. So I wonder why creation of the zip does not already throw an exception.

Do you see that file on your server?

  • Like 2
Link to comment
Share on other sites

12 hours ago, Autofahrn said:

Do you see that file on your server?

Nope, that's the strange part. The other installs run the same PW versions, same file permissions, same PHP version. 

  • Like 1
Link to comment
Share on other sites

I've just re-read the previous posts and see that CLOSING ZIP actually happens after 0.0 seconds of operation. This is pretty unlikely since I expect some time for directory enumeration and packaging the first part. Can you send me that verbose log via PM?

  • Like 1
Link to comment
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
×
×
  • Create New...