Skip to content

Commit 0dade43

Browse files
feature benchmark: update README
1 parent 7478e73 commit 0dade43

File tree

1 file changed

+3
-4
lines changed

1 file changed

+3
-4
lines changed

doc/developer/feature-benchmark.md

+3-4
Original file line numberDiff line numberDiff line change
@@ -142,11 +142,10 @@ be reported.
142142

143143
## Retry policy
144144

145-
Any suspected performance regressions will be retried up to `--max-retries` times (default is 3). Only regressions that are
146-
repeatedly reproducible will cause the benchmark to exit with a nonzero exit code. The bottom of the Buildkite log will show
147-
the retry attempts.
145+
Any scenario will be run exactly `--runs-per-scenario` times (default is 3). The run with the median wallclock duration
146+
will be chosen and other runs will be discarded.
148147

149-
Reported performance improvements are not retried to establish reprodicibility, so should be considered flukes if seen in the CI
148+
Reported performance improvements are not retried to establish reproducibility, so should be considered flakes if seen in the CI
150149
output until reliably reproduced locally.
151150

152151
# Measuring memory consumption

0 commit comments

Comments
 (0)