Skip to content

[CI] Enable Develocity test retry selectively #21479

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 28, 2024

Conversation

adpi2
Copy link
Member

@adpi2 adpi2 commented Aug 28, 2024

The sbt-develocity plugin offers the ability to retry failing tests, and to detect flaky ones. A flaky test is marked as flaky in the Develocity test report, which will help us track them across many CI executions. See for example the flaky test chart in the Pekko dashboard.

This PR contains the following configuration:

  • retry each failing test once (to be adjusted if needed)
  • don't retry if more than 10 tests fail in the current test run (to be adjusted if needed)
  • fail the build if a test is flaky
  • disable test retry in a predefined set of test classes: the compilation test classes. In those classes, we don't have enough granularity for the test retry to be meaningful, because each test is responsible for compiling many independent files.

@dotta @c00ler @Duhemm @lrytz

Test retry is disabled in all compilation test classes,
where each test is responsible for compiling many files separately.
@adpi2 adpi2 changed the title Enable test retry selectively [CI] Enable Develocity test retry selectively Aug 28, 2024
@adpi2 adpi2 marked this pull request as ready for review August 28, 2024 14:35
Copy link
Member

@sjrd sjrd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Core meeting says YES!

@adpi2 adpi2 merged commit c7c5f97 into scala:main Aug 28, 2024
28 checks passed
@tgodzik
Copy link
Contributor

tgodzik commented Aug 28, 2024

It was more like "Who wouldn't want this?" 😅

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants