Most WebP images are much bigger than their JPG sources

[rash]

Hi guys,

I’ve recently set up 2 PW installations for using WebP images (following this explanations, strategy 3). Both servers run on identical system configurations and PW versions (3.0.184). While integrating the WebP functionality was no problem at all, I’m massively confused by the results: one server works as expected, the other one does the sheer opposite.

Server 1 (the good one):
Images total: 326
WebP bigger than JPG: 41 (on average more than 40 %)
WebP smaller than JPG: 285 (on average 30–40 %)

Server 2 (the bad one):
Images total: 862
WebP bigger than JPG: 773 (on average 30–40 %)
WebP smaller than JPG: 89 (on average less than 10 %)

As far as I know, the quality of the source JPG has an impact on the WebP: highly compressed JPGs may lead to hardly smaller or even bigger WebPs,  while the savings with high quality JPGs tend to be more spectacular. Server 1 seems to confirm this assumption (the JPGs with bigger WebPs here are highly compressed 3rd party images) while server 2 ist acting completely strange. The source JPG’s size is around 1.200 x 800 pixel with a moderate compression rate and file sizes ranging between 100 and 500 kB with an average of 250 kB. The JPG quality on server 1 is about the same (regardless the 41 lousy ones), the only difference is their smaller size of 900 x 600 px with an average file size of 150 kB.

So I’d consider the WebP use on server 1 as clearly progressive, while server 2 essentially limits itself to fill up the webspace with bigger images that will never appear on a display. Is there any influence on the WebP conversion I might have missed?

[adrian]

I had a similar experience with WebP - most often not significantly smaller, and often larger, so I kinda gave up and am waiting on AVIF to be supported widely enough to start using: https://caniuse.com/avif

[rash]

@adrian Thanks for your feedback, glad to know I’m not alone. Meanwhile I did a bit more testing and found a clear correlation to the JPG compression rate. In Photoshop, a quality setting of 70 (of 100) and better leads to fairly or even much smaller WebPs, while the saving decreases with lower quality until there happens a turnaround to even bigger WebPs, approximately at 60. (The Photoshop value 70 seems to correspond to 65 in Lightroom, a bit more than 80 in Affinity Photo and something around 90 in Pixelmator Pro. The "good" JPGs on server 1 were produced by Lightroom with setting 70 and therefore match the pattern.)

All things considered, the advantage of this "next-gen format" is hard to detect. To achieve a noteworthy saving in the WebPs, you have to produce unreasonably big JPGs first, but the resulting WebP will be bigger than your more realistic old-school JPG before. Measured in figures you might have better image quality, but I doubt that the difference will be recognized by the average user. So there’s a good chance that I will join your waiting-for-avif-club.

[MoritzLost]

@rash Besides filesize, how does the visual quality of both images compare? Do the WebP images look better, worse or the same as the JPEG files? If you have a slightly larger filesize for a much better image, that might still be a good trade-off.

That said, uploading pre-compressed JPGs and then converting them to WebP is not a good idea. Re-encoding an already heavily compressed image will always result in a garbled image and larger filesize, regardless of format. You want to upload high-quality JPGs (Quality setting of 90~100) and then generate compressed variants in JPG / WebP based on the source image. In fact, you'll want to generate multiple variants in different resolutions for different screen sizes. See my tutorial on using responsive images in ProcessWire for details. This one only talks about JPG, but you can use a <picture> element with two <source> elements, one for WebP and one for JPEG.

Another thing to consider if you're seeing wildly different results on two different servers, maybe those just use different settings. WebP can encode lossless images, maybe the 'bad' server is doing that? Or if you're doing lossy compression, maybe you just need to set the WebP encoding to a lower quality level – WebP lossy encoding also has a quality setting between 1 and 100, same as JPG. There's also a range of other options to tweak to get optimal results, so maybe play with that a bit before you give up on WebP altogether. squoosh.app has a nice interface for WebP options – open a test image, select WebP in the compress menu and play around with the settings a bit. You'll be surprised at the tradeoffs between image quality and filesize you can achieve.

[Chris Ernovsky]

6 hours ago, adrian said:

I had a similar experience with WebP - most often not significantly smaller, and often larger, so I kinda gave up and am waiting on AVIF to be supported widely enough to start using: https://caniuse.com/avif

The same in our case.

AVIF or JPEG XL?

[adrian]

2 minutes ago, Chris Ernovsky said:

AVIF or JPEG XL?

Not sure, but it seems like it might be further away: https://caniuse.com/jpegxl

I feel like it's the mid 2000's again and I am waiting on browser support for SVG!

[rash]

16 hours ago, MoritzLost said:

@rash Besides filesize, how does the visual quality of both images compare? Do the WebP images look better, worse or the same as the JPEG files? If you have a slightly larger filesize for a much better image, that might still be a good trade-off.

@MoritzLost I agree. Unfortunately, it’s the other way round until now: slighty better images with considerably larger file sizes. But I’m working on it ?

16 hours ago, MoritzLost said:

 In fact, you'll want to generate multiple variants in different resolutions for different screen sizes. See my tutorial on using responsive images in ProcessWire for details. This one only talks about JPG, but you can use a <picture> element with two <source> elements, one for WebP and one for JPEG.

I already use a picture element with different sources and want to establish responsive image sizes in the next step. And I already know your tutorial and find it pretty helpful – thanks for it!

16 hours ago, MoritzLost said:

WebP can encode lossless images, maybe the 'bad' server is doing that? Or if you're doing lossy compression, maybe you just need to set the WebP encoding to a lower quality level – WebP lossy encoding also has a quality setting between 1 and 100, same as JPG. There's also a range of other options to tweak to get optimal results, so maybe play with that a bit before you give up on WebP altogether.

I’m not giving up yet, just had to whine a bit yesterday, because everything appeared so sadly.  Regarding your settings proposal: Where could I set quality levels on a shared server with limited access to it’s settings?  It‘s indeed annoying me to have either WebP or not WebP without knowing how to affect the result.

[MoritzLost]

@rash Do you use Imagick or GD to convert to WebP? For GD, the imagewebp function accepts the quality level as the third parameter. For Imagick, Imageick::setImageCompressionQuality should be the equivalent. Other options can be set with Imagick::setOption, see this post on Stack Overflow. No idea if ProcessWire's image API supports those, check the documentation or read through the source code to find out … Both of those are defined in code, so it shouldn't differ between servers. However, there are some possibilities why it might depend on the server:

• Different PHP installations with different default settings. Or different PHP versions with different levels of optimization / encoding efficiency.
• Maybe ProcessWire doesn't use GD/WebP at all, but calls the underlying C-libraries directly. In this case, the default parameters used by those libraries might be different.

[rash]

@MoritzLost I actually don’t know which component is doing the conversion, both are running on the server. As I usually tend to go the easiest way, I just rely on PW so far: calling $img->webp->url produces a WebP. So I might dive into the deeper areas of the docs as soon as I find a little patience …

The mentioned difference between the two servers results from different compression rates of JPG sources, at least that is clarified now.

[horst]

On 2/23/2022 at 3:26 PM, rash said:

Meanwhile I did a bit more testing and found a clear correlation to the JPG compression rate. In Photoshop, a quality setting of 70 (of 100) and better leads to fairly or even much smaller WebPs, while the saving decreases with lower quality until there happens a turnaround to even bigger WebPs, approximately at 60. (The Photoshop value 70 seems to correspond to 65 in Lightroom, a bit more than 80 in Affinity Photo and something around 90 in Pixelmator Pro. The "good" JPGs on server 1 were produced by Lightroom with setting 70 and therefore match the pattern.)

Hey, it seems to me that there is a wrong thinking or understanding to the whole subject that leads to the confusion. (?)

If your comparison are between the compressed image from a third party tool, e.g. photoshop or others, and ONLY the webP output from GD or imagick, then this is comparing apples with bananas.

If that's the fact, please read on. If that's not true, please excuse my post and my misunderstanding and forget / don't read the rest of the post. ?

Spoiler

It doesn't matter how much you shrink your src image before you upload it, because every image manipulation requires loading the image COMPLETLY UNCOMPRESSED into the memory. That means, if you uploaded a 1000 x 600 px image with the lowest quality and with the highest compression (e.g. filesize 10kb) OR the exact same image with highest quality and none or lowest compression, 1000 x 600 px (filesize = 2 MB), results in the exact same memory image: 600 x 800 x 8 x 3 for sRGB colored images. So, the only difference is the resulting quality of any variation image. I highly advise(d) to use one of two possible strategies for the use of images in PW (or maybe any other CMS, too), depending of your use case:

A) If you want to use your images without any transformation (resizing), means "as is", then upload them as much compressed as the visual style does allow.

B) If you want to further create variations from your uploaded image, then it is used as a master image that itself NEVER should or gets displayed to the FE. In this case the best results in all categories only can be achieved by uploading the highest possible quality (100% or quality-12 in photoshop). This is the best, because your compressing savings only have effects on uploaded bandwith/time and disk space usage, but also have visual quality loss AND, and that's real too, variations created from a 100% quality master image and the exact same 70% quality master image results in lesser file size for that derived from the 100% master. Explanation: Lossy image compression results in additional visual artefacts, which results in higher amounts of total color counts compared to the original image. When (re)loaded into memory again, for further manipulation / resizing, the compression algorithm has to deal with MORE single colors then before, and therefor the result is of higher filesize. (and lower visual quality).

Conclusion: You only can compare jpeg and webp image filesizes where the images were created from the same memory image, and with nearly the same quality settings. Otherwise its apples and bananas.

 

These are my default start settings for every new project in the site/config.php

$config->imageSizerOptions = array_merge($config->imageSizerOptions, [
    'upscaling'     => false,       // upscale if necessary to reach target size?
    'cropping'      => true,        // crop if necessary to reach target size?
    'autoRotation'  => true,        // automatically correct orientation?
    'interlace'     => true,        // use interlaced JPEGs by default? (recommended)
    'sharpening'    => 'soft',      // sharpening: none | soft | medium | strong
    'quality'       => 90,          // quality: 1-100 where higher is better but bigger
    'hidpiQuality'  => 60,          // Same as above quality setting, but specific to hidpi images
    'defaultGamma'  => -1,          // defaultGamma: 0.5 to 4.0 or -1 to disable gamma correction (default=2.0)
    'webpAdd'       => false,       // set this to true, if the imagesizer engines should create a Webp copy with every (new) image variation
    'webpQuality'   => 84           // webpQuality: 1-100 where higher is better but bigger
]);
$config->webpOptions = array_merge($config->webpOptions, [
    'quality'           => $config->imageSizerOptions['webpQuality'],
    'useSrcExt'         => true,    // Use source file extension in webp filename? (file.jpg.webp rather than file.webp)
    'useSrcUrlOnSize'   => false,   // Fallback to source file URL when webp file is larger than source?
    'useSrcUrlOnFail'   => true     // Fallback to source file URL when webp file fails for some reason?
]);

 

EDIT: maybe of additional interest: 

 

 

Edited February 24, 2022 by horst
added link to comparison tables

[rash]

@horst Thanks very much for your detailed explanations that help me quite a lot. Following your arguments, I’m indeed comparing apples with bananas. Usually I’m working with a mixture of your strategies A & B by using one image "as is" and create further JPG variations of them, mostly thumbnails, where neither quality nor file sizes are somehow critical. As I’m planning to use WebP and a broader palette of size variations, I should probably seize the opportunity to overthink my (bad) habits and switch to pure strategy B.

Embarrassingly, I never came up on the idea to use config settings for image processing, probably because I didn’t feel an urgent need for it. With strategy B they seem to be useful if not essential. So thanks for that important hint too. I will do a serious testing with all those brand new perceptions.