adrian

Thanks @flydev ?? for your hard work on this. I installed on a Debian server and had a few issues which I have managed to fix. I submitted a bit of a messed up PR - I think we were both working at the same time. All my tests were in native mode 1) The server must have the "zip" package installed - mine didn't which caused me some grief for a while. I suggest looking for this and warning if not installed. 2) I had to change the mysqldump command quite a bit - you can see in the PR, but key things were --single-transaction and specifying the DB and removing the --skip-lock-tables 3) Had to chmod the .sh script to 744 so it was executable, but obviously this may not work on all servers depending on the owner etc. 4) Had to change the directory and the exec call to actually execute the .sh file - it was failing to run otherwise. With those changes everything seems to work great, although I did notice that after running a duplication via the Process module, it no longer reloads the page when done so you don't see the new package unless you manually reload when it's finished. Thanks again!

Hi @Knubbi - can you please test the attached version of that action? If it works as expected for you, I'll commit it to the module repo. FieldSetOrSearchAndReplace.action.php

Glad that worked but I expect it will likely come back for you so it would be good to figure out what the fix will be. Please keep an eye out for it again.

Thanks @rick - don't worry about those VSCode reported errors - it just doesn't know about those classes, but they work as expected. I think it's likely that you have some non-escaped character or a character encoding issue in your log files that is causing the problem. Could try a mb_convert_encoding() on $logLinesData before it is sent to WireCache on line 44. If that doesn't work, maybe instead try utf8_encode() or htmlentities() or something else along those lines. There is this old topic: but I don't think it will help us much here.

Hi @rick - I'm not sure why that would happen but it would be helpful if you could dump the content of $data before line 449 in WireCache.php so we can see what is being passed to json_encode so we can see why it might be failing. You could/should probably also check the value of $logLinesData before line 44 in ProcesswireLogsPanel.php Please let me know what you find out.

Thank you! FYI - after setting innodb_buffer_pool_size based on that calculation query, my duplicator cron failed so I almost feel like that made things worse. I am beginning to wonder if InnoDB just needs more resources than MyISAM and my VPS server just doesn't have enough oomph ?

Thanks for the detailed info. I must admit to being pretty new to InnoDB and actually I am glad you mentioned it, because the other site with the bigger DB which is having no problems at all is on MyISAM, so I guess that is the difference. I have already played with a few of those settings you mentioned, but nothing consistently helped. The only thing that seems to be working is the change to set PHP time limit and max execution time to both infinite. It's also really confusing that this is only an issue when run via cron - it makes me think there is a significant difference in PHP or MySQL settings. Obviously PHP has a CLI specific .ini file, but I don't think there is anything like that for my.cnf. Maybe Duplicator should look into using WireDatabaseBackup's exec mode: https://github.com/processwire/processwire/blob/321ea0eed3794c5f2b50c216b603fad3e7347ce6/wire/core/WireDatabaseBackup.php#L131-L133 - any thoughts on trying that? Regarding your suggestion of disabling Tracy - I have been thinking for a while about having an option to disable Tracy when php_sapi_name() shows the script is running from the CLI. I haven't added it yet, but certainly could, although I doubt that's the cause of this issue. I am going to keep monitoring the daily cron over the next week to see if it works every day with those set_time_limit(0); ini_set('memory_limit', '-1'); settings and if it does, at least that will tell us something ? Thanks again, Adrian

Thanks for the update @flydev ?? but please don't forget about the notice caused by the undefined $zipFilename on this line: https://github.com/flydev-fr/Duplicator/blob/475318d2893e07ab225c89d738efd59d06277546/Duplicator.module#L44

Ok, I'm getting there ? @flydev - there is still one more notice in 1.3.13 - line #44 should be: $this->log("Logging {$logName}\n"); $logName, rather than $zipFilename which isn't defined. Regarding the mysql gone away error, it seems that has been fixed by doing this: //DUP_Util::setMemoryLimit(self::DUP_PHP_MAX_MEMORY); //DUP_Util::setMaxExecutionTime(self::DUP_PHP_EXECUTION_TIME); set_time_limit(0); ini_set('memory_limit', '-1'); I know this is not ideal, but it's time to move on so will have to do for now.

Thanks @Autofahrn - that new version does fix those notices. Unfortunately I am getting MySQL gone away errors again when running via cronjob. I don't expect that is to do with the new version, but rather that it's a bit random and the tests after I changed the max_allowed_packet setting didn't actually fix it. The weird thing is that the error is being triggerd ~6 seconds into the duplicator process so it's not a timeout issue.

Thanks @Autofahrn but that is the version I am using. Sorry, I take that back - I am using 1.3.12. Where do I find 1.3.13?

@flydev - sorry, just noticed something new. Even those the Duplicate Process module page shows a valid package created from the cronjob, I am seeing these logged notices that suggest the package was not built correctly. Any ideas?

@flydev - a couple of bugs and questions for you. Firstly, I am getting these notices with the current dev branch: Would be great if those could be cleaned up please. The other question is a weird one. With both the master and dev versions I have been getting "mysql gone away errors" lately on one site/server, but only from the cronjob (run from the system cron). If I do a Backup Now from the Duplicator Process module page it works fine. So it seems like there is some strange difference when run via CLI with the cronjob. The database size recently went over 128MB which is the size of my max_allowed_packet setting. I bumped it up and now it works from the command without the error. The weird thing though is that I have another server which has been working fine with a max_allowed_packet of 16M and the database is over 250MB. Both servers are Digital Ocean VPS. The one with the errors is running the latest version of Debian and the one without errors is on UBUNTU. The Debian server shows: Ver 15.1 Distrib 10.3.18-MariaDB, for debian-linux-gnu (x86_64) using readline 5.2 while the UBUNTU one shows: Ver 14.14 Distrib 5.7.28, for Linux (x86_64) using EditLine wrapper So I am wondering if it's a MariaDB vs MySQL issue, or the version difference or something else. Anyway, wondering if you (or anyone else) might have come across this recently. Thanks!

But why - I don't really understand the problem. Arrays can have keys with uppercase characters - why are we ending up with the original and an extra entry with a key that is all lowercase?

@nbcommunication - I finally figured out the extra send / count issue. We had 24 subscribers with uppercase characters entered in their email address. If I do a strtolower() on each address before building up the "to" and recipient variables arrays, everything works as expected. I am not sure if this is a bug in Mailgun, PW, or this module. Any thoughts?

Just added a new Viewports panel based on a suggestion by @bernhard in response to this post: https://processwire.com/talk/topic/22617-sizzy-browser-developer-tool/ It's pretty basic, but gives you a quick and easy way to view and interact with your site at multiple screen viewport sizes all on the one page (panel) just by scrolling down. Currently there are 6 fixed sizes, but I can expand on these or make them configurable if you'd li Take a look and let me know what you think.

@nabo - I just did some cleanup of that code and was actually also needing to link URLs, so I just use this: $text = preg_replace('/(<td>)(http(.*?))(<\/td>)/', '$1<a href="$2">$2</a>$4', $text); which is obviously much cleaner than the preg_match_all approach - I think I must have been in a rush last time and not thinking properly ?

Just a quick thought until I have more time to test. Because I don't have the "name" of the recipients, I am setting up $to in a loop like this: $to[] = $u->email; instead of: $to['email'] = 'My Name'; Do you think that having a numeric instead of associative array could be a problem?

Sorry @nbcommunication - in the middle of some other things at the moment. I actually ended up setting the "to" array because we had a newsletter going out this morning, so I will need to go back to dev setup to test without it. I'll take a look later and get back to you. I do have another question for you though. When sending, the returned value is the number of emails sent, which for this module is the returned value of the apiRequest() method. The weird thing is that in my testing with a few recipients, the count was always correct, but my client sent our first proper mailout today and she said it reported more than the number of people that we have in our list. It's only a list of 130 users at the moment, but it returned 154. Have you seen anything like this before? Do you think this is something I should ask via MailGun support? Thanks.

That actually is correct - the email field for some recipient groups is "email", but for others it's "title" - $email is defined further up. Sorry for the confusion though ?

@webcraft - as of today, 3.0.148 is the new master version (https://processwire.com/download/core/) so please use that and you shouldn't have anymore PHP 7.4 issues.

There's more than one way to skin a cat. Anyway the current version fixes it as well, just in a different way.

There are several Windows users who have it working, including @bernhard - there are posts above that detail a few different ways of getting it to work and also some more info on the docs site - I'm honestly not sure what is best.

Sorry about that @dragan - should be fixed in the version just committed. PS - glad you're enjoying Module Toolkit ?

Just in case you haven't come across this site before: https://caniuse.com/#search=webp