You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Fix float64 comparison test failure on archs using FMA
Architectures using FMA optimization yield slightly different results so
we cannot assume floating point values will be precisely the same across
different architectures.
The solution in this change is to check "abs(a-b) < tolerance" instead
of comparing the exact values. This will give us confidence that the
histogram buckets are near identical.
Signed-off-by: Seth Bunce <[email protected]>
0 commit comments