You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
There are a few tests that are disabled because the baseline comparisons fail on the 5th decimal for some of the numbers generated, on some OS.
As an example, the RegressorOlsTest() in PredictorTests fails just on the Mac debug version, because only one out of the generated 4896 predictions doesn't match:
Mac debug run predictions: 2625 5 5.091751 0.0917510986328125 0.0084182641003280878
I think we should make the tests more lenient to failures like this, modifying the comparison with the baseline to:
1- Have a sensitivity threeshold. Compare up to the 4th, or 6th decimal digit.
2- Count the failures, and declare the test as failed if 3% o the predictions/lines differ?
We can, but the downside is that sometimes, like the example above, you have an entire file in, because one line is different.
We can have a system of baselines and overrides for that baseline: if no override default to the baseline. @Ivanidzo4ka what do you think?
There are a few tests that are disabled because the baseline comparisons fail on the 5th decimal for some of the numbers generated, on some OS.
As an example, the RegressorOlsTest() in PredictorTests fails just on the Mac debug version, because only one out of the generated 4896 predictions doesn't match:
baseline:
2625 5 5.09176636 0.091766357421875 0.0084210643544793129
Mac debug run predictions:
2625 5 5.091751 0.0917510986328125 0.0084182641003280878
I think we should make the tests more lenient to failures like this, modifying the comparison with the baseline to:
1- Have a sensitivity threeshold. Compare up to the 4th, or 6th decimal digit.
2- Count the failures, and declare the test as failed if 3% o the predictions/lines differ?
@justinormont @TomFinley @Zruty0 @zeahmed are those acceptable ranges?
The text was updated successfully, but these errors were encountered: