Calculation of image difference percentage is incorrect #107

New issue

Jump to bottom

Closed

NathanielAvery opened this issue Sep 2, 2022 · 2 comments

Assignees

Labels

bug

NathanielAvery commented Sep 2, 2022

Hi 👋🏻 Thanks for writing this plugin!

Describe the bug

When the plugin determines the percentage of changed pixels, its calculation is incorrect:

cypress-plugin-visual-regression-diff/src/plugins.ts

Line 164 in b4c6617

imgDiff = (diffPixels / width) * height;

As pixelmatch returns the number of different pixels, this line should be:

imgDiff = diffPixels / (width * height);

To Reproduce

Create an empty 100x100 <div> element
Capture a snapshot of that element in a Cypress test with a threshold of 0.5.
Modify the snapshot image so that just below 50% of the pixels are different.
Re-run the test.
Expected behavior: Test passes.
Actual behavior: Test fails.

The text was updated successfully, but these errors were encountered:

NathanielAvery added the bug label

FRSgit self-assigned this

Member

FRSgit commented Sep 2, 2022

Hi @NathanielAvery,

🤦 You're absolutely right. Thanks for the good catch!

FRSgit closed this as completed in

1e9bb6d

FRSgit added a commit that referenced this issue


          fix: image diff calculation

529cb22

closes #107

Signed-off-by: Jakub Freisler <[email protected]>

Member

FRSgit commented Sep 2, 2022 •

edited

Loading

Fixed in 529cb22 (1.9.14)

github-actions bot pushed a commit that referenced this issue


          chore(release): 1.9.14 [skip ci]

124d464

## [1.9.14](v1.9.13...v1.9.14) (2022-09-02)

### Bug Fixes

* image diff calculation ([529cb22](529cb22)), closes [#107](#107)

github-actions bot pushed a commit to braze-inc/cypress-plugin-visual-regression-diff that referenced this issue


          chore(release): 1.0.0 [skip ci]

9f3104b

# 1.0.0 (2022-10-31)

### Bug Fixes

* btoa utf8 encoding/decoding error ([FRSOURCE#114](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/114)) ([0137014](0137014))
* create missing dirs when renaming screenshot files ([38e5ff5](38e5ff5))
* **deps:** pin dependency vue to 3.2.37 ([FRSOURCE#68](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/68)) ([d09a762](d09a762))
* **deps:** update dependency @frsource/base64 to v1.0.3 ([FRSOURCE#144](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/144)) ([09ecbd8](09ecbd8))
* **deps:** update dependency move-file to v3 ([FRSOURCE#62](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/62)) ([4f6eaf6](4f6eaf6))
* **deps:** update dependency pixelmatch to v5.3.0 ([FRSOURCE#55](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/55)) ([ca5d278](ca5d278))
* **deps:** update dependency sharp to v0.31.1 ([FRSOURCE#132](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/132)) ([15f0f5d](15f0f5d))
* **deps:** update dependency vue to v3.2.38 ([FRSOURCE#101](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/101)) ([e2d3c82](e2d3c82))
* **deps:** update dependency vue to v3.2.39 ([FRSOURCE#110](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/110)) ([8a7f055](8a7f055))
* **deps:** update dependency vue to v3.2.40 ([FRSOURCE#131](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/131)) ([537fd16](537fd16))
* image diff calculation ([529cb22](529cb22)), closes [FRSOURCE#107](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/107)
* proper readme info ([dd87e19](dd87e19))
* remove alias leftovers from dist bundles ([407ce79](407ce79))
* remove automated screenshots update ([acb3ef0](acb3ef0))
* reset name cache after tests run ([bfbf138](bfbf138))
* sanitize screenshot filenames ([fc57380](fc57380))
* security vulnerabilities ([d0bda44](d0bda44))
* security vulnerability ([d6f849c](d6f849c))
* text overflowing when image is small ([3b04f8e](3b04f8e))

### Features

* add forceDeviceFactor option ([8d69632](8d69632))
* add matchAgainstPath option ([FRSOURCE#146](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/146)) ([7a5e3a8](7a5e3a8)), closes [FRSOURCE#88](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/88)
* add possibility to change images dirname ([b831e94](b831e94))
* add queue flushing in after block ([70f828f](70f828f))
* add title option to matchImage command ([FRSOURCE#81](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/81)) ([4d03866](4d03866))
* add typings ([0a0e8e6](0a0e8e6))
* auto clean unused files ([FRSOURCE#124](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/124)) ([38679a7](38679a7)), closes [FRSOURCE#118](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/118)
* don't override screenshots if not needed ([9066017](9066017))
* externalize important APIs ([9f94086](9f94086))
* first implementation ([388cccf](388cccf))
* img diff when resolution differs ([FRSOURCE#108](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/108)) ([c8a5044](c8a5044)), closes [FRSOURCE#94](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/94)
* introduce imagesPath option ([FRSOURCE#152](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/152)) ([961e137](961e137)), closes [FRSOURCE#147](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/147)
* make library cypress 10 compatible ([b26beb3](b26beb3))
* make plugin Cypress 10 compatible ([a03a17d](a03a17d))
* migrate to @frsource/base64 package ([e4f3a14](e4f3a14))
* provide modern exports ([5c911a1](5c911a1))
* show comparison for successful tests ([FRSOURCE#137](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/137)) ([c09bab3](c09bab3)), closes [FRSOURCE#104](https://github.com/braze-inc/cypress-plugin-visual-regression-diff/issues/104)
* show scrollbar for overflowing images ([de994b9](de994b9))
* stop logging all of the tasks ([573e728](573e728))

### BREAKING CHANGES

* deprecate imagesDir option in favor of imagesPath - see docs for additional information
* To use autocleanup feature you need to update all of the screenshots, best do it by running your test suite with cypress env 'pluginVisualRegressionUpdateImages' set to true.
* matchImage returns object containing comparison details from now on (previously was returning subject element from a chain)
* different resolution doesn't fail test immediately - img diff is being done

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment