Pearl metrics

May 15, 2021
Image

What are pearl metrics?

Shortly put, the pearl metrics allow ShimmerCat’s image optimization platform to operate better in images that have high exposure, low contrast and uniform background, while doing just as well as the teal metrics in other images.

Explanation

Each of the WebP, AVIF, JPEG, JPEG2000 and JPEG-XL encoders can be configured to produce worse-looking but smaller images, or better-looking but bigger images. We work by ordering a subset of the settings of each encoder in a line, and assigning to each setting a value between 0 and 1.

In technical terms, we find a total ordering of each of the encoder settings with regards to bits-per-pixel

When we are optimizing an image for display in a browser, what we are actually doing is searching for some encoder settings that produce the smallest file with an acceptable amount of encoding artifacts. [In technical terms, we do half-interval search in the total ordering of encoder settings]. To control for the amount of encoding artifacts, we use something called metrics. The metrics are algorithms that look at two images and output a similarity score, with 1 for “the images are absolutely similar” and 0 for “they are completely different”. This approach allows us, for example, to serve the smallest file between all the formats that we produce and that the browser accepts, while having confidence that the images have equivalent quality.

You can think of the metrics as a quality inspector that only lets through images with sufficient quality.

Traditionally, the similarity score considers that all parts of an image are equal. That is not the case for our customers. For example, fashion e-commerce retail is most interested on having high quality in the pixels that correspond to clothing, as opposed to background or human skin. A few months ago, we introduced the teal metrics to deal with this particular scenario. These metrics allow our customers to blend well-established similarity metrics like SSIM and MSE with algorithms that give more weight to clothing. The teal metrics however don’t go all the way: while their weighted variant gives lower importance to background pixels, it doesn’t exclude them. The rationale is that some of our fashion customers put a lot of effort in art for those backgrounds.

There are cases where the background has no importance whatsoever, like in the image with the string of pearls below. It is slightly blurred, it has high exposure and a white background that fills most of it. Our job is not to comment on the art quality of said image, but to compress it with minimal quality loss.

Figure: An image with low contrast, high exposure, and predominant white background. An image with low contrast, high exposure, and predominant white background.

To our woe, uniform white color is perfectly encoded by our encoders. To make matters worse, digital displays use something called gamma-correction, that exaggerates small color variations when those colors are very bright. When our metrics, teal included, average pixel quality over the entire image, they can end up giving a very high score even to an image with bad compression quality.

Our solution to that problem is to introduce the pearl metrics. They function exactly as the teal metrics, but they ignore areas of the image that have uniform color. For the results, check image 2b. Additionally, the pearl metrics take into account gamma correction. Because the only difference between the teal metrics and the pearl metrics is which pixels they use as input, there are no changes in metric value, and thus image quality, for images that have adequate contrast and non-uniform backgrounds. This should make straightforward for our customers who currently rely on the teal metrics to switch to the pearl metrics, without needing to have the majority of their images re-optimized.

How to use

Similar to the teal metrics, the pearl metrics can be accessed in the encoder settings using the metric names pearl-ssim, pearl-ssim-mse, and pearl-ssim-mse-weighted. Or if we are managing your license, just contact our support to have it enabled for you.

Curious on how much you can benefit from ShimmerCat?

Fill the form and get a performance report, find your bottlenecks and explore possibilities.

GET PAGE TEST REPORT