JpegOptimImage

[Jonathan Dart]

Hi Guys,

I created a new module to optimize jpegs using the command line tool jpegoptim which yields amazing results, it's comparable to jpegmini.com when using 75 quality.

Instructions and usage are available in the readme on github.

https://github.com/jdart/JpegOptimImage

Let me know what you think!

Thanks

[adrian]

Hey Jonathan,

Thanks for this. I have just done some initial testing and it's working fine. I haven't yet done any comparisons with GD or Imagick to see if it is any better size/quality wise, but you sound convinced, so I am looking forward to analyzing more.

One thing I did notice is that the ".jo" files will be left behind after deletion of the initially uploaded file. It is easy to extend Pageimage::isVariation to handle these. Horst did it with the PIM module and I did it with ImageRasterizer.

EDIT - any thoughts in incorporating http://optipng.sourceforge.net/ (or similar) as well for PNGs?

[horst]

@Jonathan: looks very interesting. Haven't tested now, but have one suggestion regarding metadata: You have set "--strip-all" as default. I really would like to see that IPTC-data would be preserved if populated. This is the default behave in ProcessWire since version 2.3.0. IPTC data only is present if people explicitly have populated them, and in this case they have highly interest in keeping that information within the images and its variations. IPTC data is really small, not like EXIF or XMP. The default is to strip everything but not IPTC.

You simply can replace "--strip-all" by

       --strip-com --strip-exif --strip-icc

Or Imagesizer can / should apply a public function that adds it to the resulting files of your and other third party modules. So with jpegoptim it would be a bit faster and more straight forward to use its own flags.

------

I will test it soon, together with ImageSizer, ImagickResizer and the PiM and report back the results. 

[horst]

@Jonathan: Many thanks for this nice module. I have played around with it the whole evening.

The JpegOptim runs smoothly together with the ImageSizer and with the ImagickResizer. Following are some suggestions for you. If you find something useful, please feel free to take and implement it:

The ImageSizer supports unlimited individual options for thirdparty modules, but it isn't documented actually. You can add any assoc option to the images $options array in templates. But you need to avoid overriding the core ImageSizers options. Therefor I have prefixed all options from JpegOptim with a jo, followed by the initial option name in camelcase style. Now the options are parsed in this order: default settings, config page settings, template / image-options settings!

Example that overrides the default setting and custom setting for quality and threshold:

$options = array('joQuality'=>85, 'joThreshold'=>20);
$image->width(300, $options);

  

 

 

    

I have introduced an option that temporary suppresses the use of JpegOptim on a per image basis:

$options = array('useJPEGOPTIM'=>false);
$image->width(300, $options);

  

 

 

Also I have played around with the option for lossless optimization, but this seems to be nearly useless (not lossless ) because it produces the same result like quality 100 and threshold 1. I have left it in the code, just for documentation.

I don't have added / tested these enhancements with the pageimage::optimize hook. So if you want to use some of the above you need to test / implement it there too.

I have found out that the JpegOptim produces best results and also mostly smaller results if the GD or Imagick source is rendered with 100% quality. If you render them with e.g. 80% quality, the resulting filesize from JpegOptim is slightly higher (sometimes). This can be due to more jpeg artefacts in the result from GD or Imagick

Also it seems best solution to use strong sharpening (over sharpen the resized variation, what is the source for JpegOptim).

$options = array('quality'=>100, 'sharpening'=>'strong', 'joQuality'=>80);
$image->width(300, $options);

Different tests and results can be viewed here: http://images.pw.nogajski.de/jpegoptim/

PS: as I have explained in my previous post it would be nice to keep IPTC if populated

PPS: here is the code that helps testing: https://gist.github.com/horst-n/4757d5a34f2278657b6d#file-jpegoptim-testroutine-php

Edited June 15, 2014 by horst

[adrian]

Wow horst - just WOW!

Once again, thanks for the amazingly detailed testing and for making the results available for all of us to see in such a well formatted and documented way!

[Jonathan Dart]

Wow horst indeed! I will need some time to digest your feedback, stay tuned.

Adrian optimizing png's would be useful as well - I'll see what that would entail.

[Jonathan Dart]

EDIT - any thoughts in incorporating http://optipng.sourceforge.net/ (or similar) as well for PNGs?

Do you have any thoughts on which png tool is the best? I found the below resources but I'm not sure what to make of it all, I guess it depends if we want lossy or lossless png compression.

http://pointlessramblings.com/posts/pngquant_vs_pngcrush_vs_optipng_vs_pngnq/

http://css-ig.net/png-tools-overview.html

[adrian]

Sorry Jonathan - I have played with some of these tools a bit in the past, but never really figured out which were best so I can't really offer any advice. I guess ideally I'd like to see options for lossless and lossy. It's strange I have always thought of PNG as a lossless format, but I know that things have changed on that front. 

[horst]

@Jonathan: Hi, it's me again, - I have some more thoughts / suggestions

If someone want to use your optimize module and simply the ImageSizer (or any replacement of it), it's just fine as is. But if someone want to use the PageimageManipulator and the Optimizer, the option to temporary suppress the Optimizer is needed:

$image->pimLoad('prefix')->someManipulations()->pimSave()->width(300); // this works fine, no need to suppress the optimizer

$image->width(300)->pimLoad('prefix')->someManipulations()->pimSave(); // this doesn't work, we need to suppress the optimizer in the width(300) call!
// we only want to optimize the final variation and don't want optimize intermediate images

 

 

 

Next points all regard to   pageImageOptimize()  /  pageimage::optimize():

With the initially module, the pageimage::optimize() was made for original images only. To preserve the original source (for quality etc) you create a variation with a name like basename.jo.jpg. That's all perfect. There is only a little issue: with the .jo in the name of the variations they get not fetched by the pageimage variation collections. As a result, when deleting / removing original images, these variations would stay as orphaned files. So, I know there is currently no way provided by PW to solve that   (But there is already a private discussion with Ryan on how to provide a good support for custom imagefilename suffixes)

As a quick (but bit clumpsy) workaround, you can change the .jo by a .0x0 - This matches the current used regex of the variation collections and isn't a name that could already exist.

 

Not only after the use/need for the temporary suppression of the optimizer, there are application cases where someone need / want to call pageimage->optimize() not only on original images. For example when working with the PageimageManipulator:

$image->pimLoad('prefix')->someManipulations()->pimSave()->optimize(); // here simply the existing file should be optimized, no need for a new variation name or a new pageimage object

I have updated the pageimage::optimize() to recognize original images vs variations and act upon that.

 

If the basic usage supports individual options per image, the optimize hook needs too:

$options = array('joQuality'=>80, 'joThreshold'=>30);
$image->optimize($options);

-----------------

Finally some thoughts about how the different Pageimage modules could work together

- If you implement support for PNG too, a name change to something like PageimageOptimize  (PageimageOptimizer) would be useful.

- Could it be useful for the average user that you do a check against the ImageSizer quality (hook)-before resizing? And if there is no individual setting, temporary set it to 100%? This way an average user do not have to know all about how it works best. And all users can save some code writing, no need for:

$options = array('quality'=>100);
$image->width(300, $options);

just type $image->width(300) and get the best possible result.

- with the PageimageSizerImagick (formerly known as ImagickResizer), the user can choose what type of metadata he want to keep, EXIF, XMP, ICC-profile. XMP isn't supported by JpegOptim, but the others too. Would it be useful to support these with the Optimizer? I can imagine that both modules have their own global settings in their config screens, but both check for individual settings on a per image basis:

$options = array('keepEXIF'=>false, 'keepICC'=>true);
$image->width(1200, $options);

$image->optimize($options);

I think their will be some more Sizers in the future, PageimageSizerImageMagick, PageimageSizerNetpbm, PageimageSizerWhatever and all of them should support and use the same options where possible. This way a user can write code for image handling independent of the image rendering engine.

What do you think about this? I know its much at once and I have a bit of a guilty conscience to push so much stuff here. But I think it is needed that the existing and future image modules work well together and just by installing / uninstalling them.

[muzzer]

Looks cool, but am I correct in saying you need shell access on your hosting server to use this module as you need to install jpegoptim?

[horst]

On many hosts / shared hosts this is installed already. If not, you need shell access or need to ask your hoster to install it.

But another thing is that you need PHP with enabled / allowed exec() function. In shared environments this function mostly is disabled / forbidden if php runs as a apache module, but is allowed if php is running as (fast)cgi.

Maybe a check for exec() should be done by installation.

[Jonathan Dart]

@Jonathan: Hi, it's me again, - I have some more thoughts / suggestions

If someone want to use your optimize module and simply the ImageSizer (or any replacement of it), it's just fine as is. But if someone want to use the PageimageManipulator and the Optimizer, the option to temporary suppress the Optimizer is needed:

$image->pimLoad('prefix')->someManipulations()->pimSave()->width(300); // this works fine, no need to suppress the optimizer

$image->width(300)->pimLoad('prefix')->someManipulations()->pimSave(); // this doesn't work, we need to suppress the optimizer in the width(300) call!
// we only want to optimize the final variation and don't want optimize intermediate images

 

 

 

Next points all regard to   pageImageOptimize()  /  pageimage::optimize():

With the initially module, the pageimage::optimize() was made for original images only. To preserve the original source (for quality etc) you create a variation with a name like basename.jo.jpg. That's all perfect. There is only a little issue: with the .jo in the name of the variations they get not fetched by the pageimage variation collections. As a result, when deleting / removing original images, these variations would stay as orphaned files. So, I know there is currently no way provided by PW to solve that   (But there is already a private discussion with Ryan on how to provide a good support for custom imagefilename suffixes)

As a quick (but bit clumpsy) workaround, you can change the .jo by a .0x0 - This matches the current used regex of the variation collections and isn't a name that could already exist.

 

Not only after the use/need for the temporary suppression of the optimizer, there are application cases where someone need / want to call pageimage->optimize() not only on original images. For example when working with the PageimageManipulator:

$image->pimLoad('prefix')->someManipulations()->pimSave()->optimize(); // here simply the existing file should be optimized, no need for a new variation name or a new pageimage object

I have updated the pageimage::optimize() to recognize original images vs variations and act upon that.

 

If the basic usage supports individual options per image, the optimize hook needs too:

$options = array('joQuality'=>80, 'joThreshold'=>30);
$image->optimize($options);

-----------------

Finally some thoughts about how the different Pageimage modules could work together

- If you implement support for PNG too, a name change to something like PageimageOptimize  (PageimageOptimizer) would be useful.

- Could it be useful for the average user that you do a check against the ImageSizer quality (hook)-before resizing? And if there is no individual setting, temporary set it to 100%? This way an average user do not have to know all about how it works best. And all users can save some code writing, no need for:

$options = array('quality'=>100);
$image->width(300, $options);

just type $image->width(300) and get the best possible result.

- with the PageimageSizerImagick (formerly known as ImagickResizer), the user can choose what type of metadata he want to keep, EXIF, XMP, ICC-profile. XMP isn't supported by JpegOptim, but the others too. Would it be useful to support these with the Optimizer? I can imagine that both modules have their own global settings in their config screens, but both check for individual settings on a per image basis:

$options = array('keepEXIF'=>false, 'keepICC'=>true);
$image->width(1200, $options);

$image->optimize($options);

I think their will be some more Sizers in the future, PageimageSizerImageMagick, PageimageSizerNetpbm, PageimageSizerWhatever and all of them should support and use the same options where possible. This way a user can write code for image handling independent of the image rendering engine.

What do you think about this? I know its much at once and I have a bit of a guilty conscience to push so much stuff here. But I think it is needed that the existing and future image modules work well together and just by installing / uninstalling them.

Hi Horst,

I reviewed most of your changes and integrated them:

->optimize now accepts an array options so you can customize behavior per call

->optimize won't overwrite original files, a variation would be created

->optimize will overwrite a variation

I had a look at passing the options to size() with the purpose of them getting passed on to this module, but it seems that if the options aren't the usual ones used by ImageSizer they get discarded, so I can't read them - which is a bummer. Do you know of a way around that? Or maybe that's changed in the dev branch?

Also regarding setting the quality to 100 before imagesizer runs if the optimizer would be used later - that would be awesome but I don't see a hook for that - do you have any thoughts on that?

[horst]

Hi Jonathan,

many thanks for adding this and your help to make all image related modules work smoothly together.

Passing third-party-options along with images is a very new feature. It has changed in the current dev-branch if I remember right around 2.4.2 or 2.4.3. Unfortunately 2.4.0 doesn't support third-party options like the current dev-branch does. With the 2.4.0 you only can get / set the original quality for the imagesizer, but nothing more.


With the comming soon 2.5.0 or current dev-branch, one way to get / set the imagesizer options (including our third-party ones) is to add another hook to the init function:

    $this->addHookBefore('ImageSizer::resize', $this, 'imageSizerResizeBefore');

And in the imageSizerResizeBefore method you can get / modify / set all options. The first thing should be to look if optimize should be skipped, if so, one can set a flag that will be read first by the imageSizerResizeAfter method too.

public function imageSizerResizeBefore($event) {

    $imageSizer = $event->object;
    $this->skipOptimze = (false===$imageSizer->useJPEGOPTIM) || ($imageSizer->getImageType() !== IMAGETYPE_JPEG) ? true : false;

    if($this->skipOptimze) return;  // this one can be called in the [font=courier new,courier,monospace]imageSizerResizeAfter[/font] method too

    // read all options and settings from the ImageSizer-Object
    $this->filename = $imageSizer->filename;
    //$this->extension = $imageSizer->extension;
    //$this->sharpening = $imageSizer->sharpening;
    //$this->upscaling = $imageSizer->upscaling;
    //$this->cropping = $imageSizer->cropping;
    $this->quality = $imageSizer->quality;
    $this->keepICC = $imageSizer->keepICC;
    $this->keepEXIF = $imageSizer->keepEXIF;

    // modify options for the resizing
    $imageSizer->quality = 100;
    $event->object = $imageSizer; // is this all what is needed to set, or do we need to set a $event->return too 

}

The above code isn't tested, just written in the browser and parts copy-pasted from another module of mine.

The PR for supporting read / write customer suffixes with image variation names is pending. Looks like it will be added soon (before releasing 2.5.0)
 

I there is more you need to know or I can do, please just ask.

[jacmaes]

This module rocks! Thanks Jonathan. I'm seeing up to 50% reductions in file size with no visible quality difference. 

[OllieMackJames]

Does this still work ok?

I am on pw 2.7.3

thanks!

[jacmaes]

@OllieMackJames, I just double-checked. I'm also on 2.7.3, and I am still seeing reductions between 35% and 50% depending on the image. 

[OllieMackJames]

@jacmaes thanks for that, will install and post back