Immediate timeout exception iMagick imageSizer

[Mikie]

I'm getting very fast timeouts on my local environment when resizing large amounts of images. Environment is php 7.3.5, PW 3.0.123, imagick 3.4.3. 

Imagick and creating variations does work, but on a page with > 10 images or so it will fail after about 1 second of execution and 10 image-sizer success logs with:  Fatal Error Maximum execution time of 120 seconds exceeded. Call stack traces to line 364 in wire/core/ImageSizerEngine.php

$options = array_merge($this->wire('config')->imageSizerOptions, $options);

I feel like this is something wrong with my local environment, but just thought I'd check if anyone has experience anything like this.

[Mikie]

Definitely my dev environment (which is homebrew apache, php, mariadb on macOS) as on staging server things are working fine. Will update if have more info.

[Mikie]

Two apache processes running (system and homebrew), I must have launched the system one again by accident somehow. Anyway, solved and solution here for posterity!

Above was false flag sorry. Even setting max_execution_time to 0 in php.ini isn't working.

Only thing that works is to also set max_input_time to -1. This is for scripts that run for about 10 or 15 seconds.

Might be time to set up docker again.

 

[Autofahrn]

You may try set_time_limit in your script:

https://www.php.net/manual/en/function.set-time-limit.php

[horst]

38 minutes ago, Autofahrn said:

You may try set_time_limit in your script:

https://www.php.net/manual/en/function.set-time-limit.php

We already have this in the image sizer loops:

https://github.com/processwire/processwire/blob/bbd3aba191876b2e8aec913357b8b687e1f441a1/wire/core/ImageSizerEngine.php#L1064-L1090

Its called here, with every image starting:
https://github.com/processwire/processwire/blob/bbd3aba191876b2e8aec913357b8b687e1f441a1/wire/core/ImageSizerEngine.php#L386

 

@Mikie : Seems that your php setup do not allow changes to max execution time! 
You can do a quick test with something like:

$old = (int)ini_get('max_execution_time');
set_time_limit(2 * $old + 5);
$new = (int)ini_get('max_execution_time');
var_dump(['old' => $old, 'new' => $new, 'supported' => (2 * $old + 5 == $new)]);

 

For troubleshooting you may double check that you edited the right(!) php.ini file,

you may check if you have 'set_time_limit' in the disallowed functions, or if you are running in safe-mode,

(look into your phpinfo() output)

[Autofahrn]

6 minutes ago, horst said:

We already have this in the image sizer loops

wasn't aware of that... ?

[Mikie]

Thanks @horst

For clarity, setting max_execution_time is seemingly working, in that if set to 0 I get "Maximum execution time of 0 seconds exceeded".

I have checked a bunch of the things you mentioned, but will double check, there is definitely something amiss.

I read on some website earlier today (can't find it now, will hunt down the link) that there can be some buggy behaviour with this setting in php, and that other services can cause it like the server (apache has a max execution time as well, default 300).

Also noticing that imagemagick is 1000% maxing all 6 of my cpu's on these transforms, although I guess that's what they are there for.

Edit: link mentioned above http://bafford.com/2016/12/02/php-misleading-error-maximum-execution-time-of-0-seconds-exceeded/

[ryan]

I'm running into this issue on my localhost (MAMP) where it reports PHP’s standard "Fatal Error: Maximum execution time of 120 seconds exceeded" before even 1 second occurs. It does this regularly, but not all the time. But if I'm clicking around pages, I'll see often (maybe 1 out of 7 requests), and it halts the request. The file reference is always somewhere in either ImageSizerEngineIMagick or ImageSizerEngine, though a common place is on the line that instantiates the IMagick instance in the processResize() method:

// start image magick
$this->im = new \IMagick();

I looked into the ImageSizerEngine::setTimeLimit() method in detail, but there's no problem I can spot in there. I have no idea what the issue could be, but just replying here since @Mikie also ran into it, so it seems to be reproducible. Though it's a weird anomaly that we should never see unless 120 seconds actually passed (but they didn't, not even close). I haven't observed it occurring on any live servers yet, just my localhost environment on PHP 7.2.10. I'm guessing the issue might be PHP version specific or IMagick version specific since it appears to be PHP outputting an error with false information. Posting here in case anyone else has insight on what it might be.

[Mikie]

Hey @ryan just hit this again and done some more research, from what I can tell there are longstanding problems on mac with the combination of apache threads, imagemagick, and the parallelization library openmp that magick uses. I am fairly certain installing imagemagick with the flag --disable-openmp should fix things. My problem is that in my setup imagemagick is installed via Homebrew and since it is a core formula I can't provide options on install anymore. I don't want to deal with installing a binary etc, this is pretty frustating since now I might need to create and maintain a tap of imagemagick with openmp disabled just to fix this.

[Mikie]

Some more info on what I am talking about:

https://github.com/Imagick/imagick#openmp
https://stackoverflow.com/questions/8413868/imagemagick-thumbnailimage-maximum-execution-time-of-30-seconds-exceeded
http://www.daniloaz.com/en/high-cpu-load-when-converting-images-with-imagemagick/
https://www.imagemagick.org/discourse-server/viewtopic.php?t=16453&start=15#p71389

It is interesting because the maintainer of php Imagick suggests images shouldn't even be processed inside a webserver due to these threading issues: https://github.com/Imagick/imagick/issues/263

The only thing that has worked for me so far in my setup (Homebrew installed imagemagick and pecl installed imagick) is adding:

\Imagick::setResourceLimit(\Imagick::RESOURCETYPE_THREAD, 1);

to my site config.