Not enough memory to load/resize

[matjazp]

When adding pictures to images field, i get the error "not enough memory to load/resize". It was working fine on 2.4.0, now after upgrade to 2.5.4 error occurs. File size or picture dimension is not relevant. Picture is uploaded, thumbnail is created, but the size of thumbnail is 1 kb and can not be opened with any picture viewer.

[horst]

Can you bump up the available memory for PHP?

It is explained here in detail: https://processwire.com/talk/topic/7749-max-image-dimensions-not-a-recognized-image-bug/

So as the factor setting actually in PW is 2.5, I will ask Ryan to set it to 2.2 by default.

Edit: have sent a question to Ryan: https://github.com/ryancramerdesign/ProcessWire/issues/739

Edited October 15, 2014 by horst

[matjazp]

Thx. I changed $faktor form 2.5 to 2.0 in wire\ImageSizer.php but error is still present.

[horst]

Thx. I changed $faktor form 2.5 to 2.0 in wire\ImageSizer.php but error is still present.

Did you have deleted the images from the previous Errors?

(Where you have no Thumbnail but the Message "Not enough memory to load/resize"?)

You need to

• delete those in your EditPage-Screen,
• save the Page
• and upload them again.

If this doesn't work, then you definitely need to bump up memory for PHP.

[matjazp]

Yes, I have deleted images from previous error (assets/pageid directory is empty). memory_limit = -1 was already set before upgrade.

[horst]

Yes, I have deleted images from previous error (assets/pageid directory is empty).

Good!

memory_limit = -1 was already set before upgrade.

Don't know for what it is good in this regard, but it also do not bother.

Now the final question: How much memory is available for PHP?

[matjazp]

I tested with 

<?php
 $step = 1;
 while(TRUE) {
  $chunk = str_repeat('0123456789', 128*1024*$step++);
  print 'Memory usage: '. round(memory_get_usage()/(1024*1024)) . 'M<br />';
  unset($chunk);
 }
 ?>

It goes over 400 MB (in task manager, i'm running PW on IIS, no problem so far) before script times out.

[cstevensjr]

   It goes over 400 MB (in task manager, i'm running PW on IIS, no problem so far) before script times out.

 

That may be a great test, however what are the PHP.ini limits set at?  The PHP.ini memory settings will have an effect on why you're having your problem, no matter what you may be able to do with on that test.  Windows and IIS has it's quirks. 

[matjazp]

What php limits do I have to look at? Or should I attach my php.ini? Why there were no problems on 2.4.2?

[LostKobrakai]

In your php.ini look for the value for this key: memory_limit.

[matjazp]

As I already pointed out, memory_limit = -1

[horst]

Ok, now I got it.

It is a bug. For now you can

A) set the value for memory_limit to 512M or 1024M or 1G (but not greater than 2G!!) if this is possible for you.

or B) If this is not possible for you, you can change this line to be:

if($phpMaxMem <= 0) return null; // we couldn't read the MaxMemorySetting, so we do not know if there is enough or not

I will file an issue at Github that fixes the setting with -1.

Sorry for the inconvenience, I haven't recognized that setting to -1 in the post above:

Don't know for what it is good in this regard, but it also do not bother.

[matjazp]

I went for option B and voila Thank you very much, Horst!

[Kurbel]

Hi Guys,

i think I might have run into the same Issue.

I've build an AJAX Uploader that returns a thumbnail for a larger Image by attaching the image to a post and giving back $page->image_field->height(200)->url. However with increasing file sizes at some point there's only invalid images getting generated on the live system (externally hosted, so no direct control over php settings).

php ini_get('memory_limit')) shows me 64M on the live system. When doing this on my local DEV-System peak memory usage is around 11MB and it works. So in theory everything should be fine on LIVE as well.

getimagesize() for the image in question returns:

{
	"0":3543,
	"1":2365,
	"2":2,
	"3":"width=\"3543\" height=\"2365\"",
	"bits":8,
	"channels":3,
	"mime":
	"image\/jpeg"
}

When I just have checkMemoryForImage() always return true, no matter what the actual calculation, the images get resized without any problem. Obviously this is not a solution that I'm very happy with, since the client will probably just feed the uploader images until php crashes against some kind of other memory limit.

However right now I need to makes this work at least for images around these specs. So I guess the question is how do I make this work without running the risk of a catastrophic failure.

Cheers.

[horst]

Hey @kurbel: I guess you use PW 2.5 stable? If you can upgrade to the dev version all related to this is fixed!

To upgrade there is a new module available here: https://processwire.com/talk/topic/7525-module-processwire-core-upgrade/

It works just fine.

[martind]

hi,

we have a similar problem here with the checkMemoryForImage() function on 2.6.10 dev.

upload_max_filesize 50M

post_max_size 50M

memory_limit 1G

... on a 45mb jpg the - not enough memory to load error occurs. Any ideas?

greetings,

martin

[horst]

Hi Martin,

if the filesize limit is 50 MB, the post_max_size should be a bit higher. I think it has to do with that post data is encoded so that the final post data is always greater than the original binary file data.

Can you try upload_max_filesize 50M and post_max_size 65M?

[martind]

hello Horst,

thank you for this, but no success with post_max_size 99M too. It seems to be a memory problem, there iss Zend Memory Manage installed on this php 5.4.42 build, but think thats pretty common, shouldn`t set any hardlimit.

[martind]

ok, have looked inside a bit. The upload itself works without problems, as long as the preview-thumbnail option isn't set for the field.

the calculation for imgmem in checkMemoryForImage() for the said 45mb image gets 1,39 gb (based on 21600x21600x3), so setting memory_limit to 2G clears this error.

But, when uploading the 45mb jpeg, the process goes quickly up to 99%, sometimes it stops a bit earlier and restarts itself again, but it comes to 99%. Then, a corrupt jpeg, titled by the sanitized source-filename, gets created, its only about 17 Mb (a quick check in photoshop shows the right dimensioned image but only the first third of the pixels are viewable). A few seconds later, a second jpeg with the original file-size of 45mb is created, titled by the sanitized source-filename and -1, this is bug-free.

So, from the side of the upload process everything seems to work fine, the problem occurs with imageSizer when calling imagecopyresampled the first time.

[LostKobrakai]

Maybe I'm wrong about this, but why would one try to resize such a large image with php? I doubt this being efficient in any way.

[martind]

for a common website case you are right, but here its a combined workflow for print/web publishing from a pr-agency, image data comes directly from organizers and phtographers and it is desired that they submit it in the highest possible resolution and quality. This works pretty well in most cases, but the time of 30+megapixel cameras brought up new requirements here.

[LostKobrakai]

That's still something I wouldn't necessarily let php do, but a dedicated image processor on the server. Otherwise maybe have a look at this: https://processwire.com/talk/topic/9814-client-side-image-resizing/ Move the resizing work to the client and submit the original and a reasonable sized image. Save the original in a file field and just use the second one to generate thumbs.

[horst]

Martin,

I would go with a fileupload field, not with an image field. If this has to do with the project we have talked about, you can copy / create a new inputfield of the default file upload field or use the default file-field and hook into file_added to further process the uploaded images with e.g. imagick or equivalent software on the server.

PS: @Lostkobrakai, you are mainly right with what you say here, but I would avoid any clientside imageprocessing and only let the users upload the high quality file because there is a good chance that it will be a mix of RGB and CMYK files (maybe grayscaled too) each colorspace with different ICC-Profiles, what I think isn't handled by clientside imageuploaders.

Edited August 12, 2015 by horst

[martind]

hello horst,

thank you for the input. What i have now is:

- this snippet that converts also really large files to needed downscaled versions without heavy load and pretty quickly. unfortunately your PageimageSizerImagick module won`t do this that way, it runs in heavy load and timeouts as with gd.

	$imagick = new Imagick();			
	$imagick->setOption('jpeg:size', '800x532');
	$imagick->readImage("/var/www".$page->test_img->first()->url);
	$imagick->thumbnailImage(800 , 532 , TRUE);
	$imagick->stripImage();
   $imagick->writeImage("/var/www".$page->test_img->first()->url."-imagick.jpg");
	$imagick->destroy();

- and further the idea to hook into a file-field and execute this after InputfieldFile::fileAdded or whatever meets best; a kindly info about the existing versions of the image and a link to recreate it for every file in the field would be sweet.

... and now i'm on the road to create a buggy module that does this. we will see ...

[horst]

Hi Martin,

uhm yeah, the imagickSizer is very old. It doesn't work with PW versions other than PW 2.4.x There were many changes done in PW core after I have created it and I haven't found the time to write a new one. Also there is an issue with transparent and / or indexed PNGs with IMagick that makes it very uncomfortable because exactly some issues with transparent PNGs in GD was the most annoying point why we once have started to build this as an alternative!