Skip to content

Run all found test frameworks, rather than just one #3621

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged

Conversation

Gedochao
Copy link
Contributor

@Gedochao Gedochao commented Apr 7, 2025

Fixes #3620

What this does:

  • on the JVM, all test frameworks should be detected and ran automatically
  • on Scala Native & Scala.js platforms, all recognised test frameworks should be detected and ran automatically
    • unknown (non-recognised) and custom test frameworks are limited to 1 at a time, and not together with others; this is due to deep-searches for unknown test frameworks being very time-consuming;
    • this is still a straight-up upgrade over the past behaviour, when a single test framework would be run at all times.
  • the --test-framework CLI option and //> using testFramework using directive work as before, where only the specified test framework is to be run
    • note that this gives the best performance, particularly for Scala Native & Scala.js tests, as this effectively skips test framework detection and runs with what is passed

Potential future follow-ups:

@Gedochao Gedochao changed the title Run all found test frameworks, rather than just one [WIP] Run all found test frameworks, rather than just one Apr 7, 2025
@Gedochao Gedochao force-pushed the feature/run-multiple-test-frameworks branch 3 times, most recently from dc34fde to 1bb63c7 Compare April 8, 2025 10:25
@Gedochao Gedochao added the needs-minor-release This change should require a minor version bump. label Apr 8, 2025
@Gedochao Gedochao added this to the v1.8.0 milestone Apr 8, 2025
@Gedochao Gedochao force-pushed the feature/run-multiple-test-frameworks branch 7 times, most recently from 1d59702 to 8d0120c Compare April 10, 2025 12:50
@Gedochao Gedochao force-pushed the feature/run-multiple-test-frameworks branch from 7545297 to 60d5362 Compare April 10, 2025 14:29
@Gedochao Gedochao changed the title [WIP] Run all found test frameworks, rather than just one Run all found test frameworks, rather than just one Apr 14, 2025
@Gedochao Gedochao marked this pull request as ready for review April 14, 2025 09:00
@Gedochao Gedochao requested a review from tgodzik April 14, 2025 09:00
Copy link
Member

@tgodzik tgodzik left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! I just added some comments, but feel free to ignore me.

@Gedochao Gedochao merged commit 1a5443a into VirtusLab:main Apr 24, 2025
54 checks passed
@Gedochao Gedochao deleted the feature/run-multiple-test-frameworks branch April 24, 2025 07:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs-minor-release This change should require a minor version bump.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

test should warn when multiple test frameworks are on the classpath
2 participants