Skip to content

Add perceptualTolerance parameter to Diffing+bitmap to compare perceptual differences using Delta E 1994 #11

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 26 commits into from
Nov 15, 2024

Conversation

JoelWhitney
Copy link
Contributor

Differences from swift-snapshot-testing

Similar to what was implemented in swift-snapshot-testing, with some slight differences

To match the established pattern of tolerance, this PR adds an optional perceptualTolerance parameter that allows you to specify the percentage each pixel can be different from source pixel and still considered a match. This parameter complements the existing tolerance parameter that determines the percentage of pixels that must be considered matching in order to consider the whole image matching.

The default value of 0.0 means pixels must match perfectly (and performs existing equality check). Any other value greater than 0.0 and less than or equal to 1.0, will be used when comparing each pixel where 0.0 (0%) means no difference and 1.0 (100%) means completely opposite. A suggested value of 0.01-0.02 is similar to the precision of the human eye.

Perceptual tolerance value Description
0.0 Must be exactly equal
≤ 0.01 Allows differences not perceptible by human eyes
≤ 0.02 Allows differences possibly perceptible through close observation
≤ 0.1 Allows differences perceptible at a glance
≤ 0.5 Allows differences when more similar than opposite
≤ 1.0 Allows any differences

Unlike swift-snapshot-testing, which uses a CILabDeltaE CoreImage Filter, we must do a byte-by-byte comparison, calculating Delta E values as on-the-fly.

Formula resource: http://www.brucelindbloom.com/index.html?Eqn_XYZ_to_Lab.html

Description of pointfreeco/swift-snapshot-testing#628...

Problem

The existing image matching precision strategy is not good at differentiating between a significant difference in a relatively small portion of a snapshot, and imperceivable differences in a large portion of the snapshot. For example, these snapshots below show that a 99.5% precision value fails an imperceivable background color change while allowing noticeable changes (text and color) to pass:

Reference Fails Passes
 image  image  image
  Imperceivable background color difference Significant text and color changes

Solution

This PR adds a new optional perceptualPrecision parameter to image snapshotting which determines how perceptually similar a pixel must be to consider it matching. This parameter complements the existing precision parameter that determines the percentage of pixels that must be considered matching in order to consider the whole image matching.

This approach is similar to pointfreeco/swift-snapshot-testing#571 and pointfreeco/swift-snapshot-testing#580 but uses perceptual distance rather than Euclidean distance of sRGB values. This is significant because the sRGB color space is not perceptually uniform. Pairs of colors with the same Euclidean distance can have large perceptual differences. The left and right colors of each row have the same Euclidean distance:

image

@JoelWhitney JoelWhitney changed the title Add param to Diffing+bitmap to allow perceptualTolerance between pixels using Delta E 1994 Add perceptualTolerance parameter to Diffing+bitmap to compare perceptual differences using Delta E 1994 Oct 24, 2024
@JoelWhitney
Copy link
Contributor Author

Hey @balazstothofficial, I know there hasn't been much activity here recently, is this PR better directed towards my fork?

@balazstothofficial
Copy link
Contributor

@KlausNie @Numoy Maybe one of you can take this over?

@KlausNie
Copy link
Member

@JoelWhitney sorry for the delay
and thank you, @balazstothofficial, for tagging me, I'd have missed it!

@JoelWhitney, I've set @eldrig0 on it, and he's getting into the topic with the intention of getting it merged!

@JoelWhitney
Copy link
Contributor Author

That's great, thank you @KlausNie @eldrig0!

I was able to test the changes in a test framework where I had a reproducible case. I meant to circle back and add unit tests but never got to it. I'll see if I can add some tests while the changes are being reviewed

@JoelWhitney JoelWhitney requested a review from eldrig0 November 15, 2024 15:24
@eldrig0 eldrig0 merged commit 7355432 into QuickBirdEng:master Nov 15, 2024
@JoelWhitney
Copy link
Contributor Author

Hey @eldrig0 @KlausNie, is there going to be a new release published following this merge?

@eldrig0
Copy link
Contributor

eldrig0 commented Nov 20, 2024

Yup, will do it tomorrow

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants