The tests passes even when it is over threshold.

[vazsonyidl]

Hi there,

if (options.skipFailure === false) {
      assert(misMatch <= options.tolerance, "Screenshot does not match with the baseline " + baseImage + " when MissMatch Percentage is " + misMatch);
    }

As the previous code snippet shows, since the options.skipFailure is undefined, the comparison would always return false. Because of this, the assert would never run, and the pipeline passes even when the tolerance is clearly over the threshold.

Please define false as default, because previously the assert run automatically, did not depend on the value of skipFailure.

[puneet0191]

@vazsonyidl could u please send fix, thanks for reporting, I will merge and make a new release.

[eunicekokor]

@vazsonyidl @puneet0191 I am also running into this, there is also no mismatch percentage shown even though there is a new image generated in diff

[vazsonyidl]

@eunicekokor i will fix it tomorrow

[puneet0191]

Thanks a lot for helping 😊

[vazsonyidl]

I do not have permission to contribute for this project. I am unable to push for a new branch created by me, so the solution should be.

if (options.skipFailure === false) { assert(misMatch <= options.tolerance, "Screenshot does not match with the baseline " + baseImage + " when MissMatch Percentage is " + misMatch); }

should be replaced with:

if (!options.skipFailure) { assert(misMatch <= options.tolerance, "Screenshot does not match with the baseline " + baseImage + " when MissMatch Percentage is " + misMatch); }

This fixes this issue. This will assert de misMatch and tolerance by default, as it should behave. So the test will not passes if the mismatch is over tolerance. Please fix this. 💯

Thanks in advance!