Skip to content

test should warn when multiple test frameworks are on the classpath #3620

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
scarf005 opened this issue Apr 6, 2025 · 4 comments · May be fixed by #3621
Open

test should warn when multiple test frameworks are on the classpath #3620

scarf005 opened this issue Apr 6, 2025 · 4 comments · May be fixed by #3621
Assignees
Labels
enhancement New feature or request test command UX

Comments

@scarf005
Copy link
Contributor

scarf005 commented Apr 6, 2025

Version(s)
Scala code runner version: 1.7.1
Scala version (default): 3.6.4

Describe the bug

adding //> using toolkit default to scalatest file

To Reproduce

main.test.scala

//> using test.dep org.scalatest::scalatest::3.2.19

import org.scalatest.flatspec.AnyFlatSpec

class ExampleSpec extends AnyFlatSpec:
    "example" should "work" in { assertResult(1)(1) }
scala --cli-version 1.7.1 test .
Compiling project (test, Scala 3.6.4, JVM (23))
Compiled project (test, Scala 3.6.4, JVM (23))
ExampleSpec:
example
- should work
Run completed in 221 milliseconds.
Total number of tests run: 1
Suites: completed 1, aborted 0
Tests: succeeded 1, failed 0, canceled 0, ignored 0, pending 0
//> using test.dep org.scalatest::scalatest::3.2.19
//> using toolkit default

import org.scalatest.flatspec.AnyFlatSpec

class ExampleSpec extends AnyFlatSpec:
    "example" should "work" in { assertResult(1)(1) }
scala --cli-version 1.7.1 test .
Compiling project (test, Scala 3.6.4, JVM (23))
Compiled project (test, Scala 3.6.4, JVM (23))

Expected behaviour

importing toolkits shouldn't prevent scalatest from working silently.

@scarf005 scarf005 added the bug Something isn't working label Apr 6, 2025
@Gedochao
Copy link
Contributor

Gedochao commented Apr 7, 2025

Now, this is actually expected.
Scala Toolkit includes Munit, while the reproduction also imports ScalaTest.
As a result, there are 2 test frameworks on the classpath and Scala CLI doesn't know which one it should use.
You can state the test framework explicitly, like this:

//> using test.dep org.scalatest::scalatest::3.2.19
//> using toolkit default
//> using testFramework org.scalatest.tools.Framework

import org.scalatest.flatspec.AnyFlatSpec

class ExampleSpec extends AnyFlatSpec:
    "example" should "work" in { assertResult(1)(1) }

That being said, I would agree a warning should be raised in such a situation, possibly mentioning the test frameworks available in the project.

@Gedochao Gedochao changed the title scalatest stops working when toolkit is used test should warn when multiple test frameworks are on the classpath Apr 7, 2025
@Gedochao Gedochao added enhancement New feature or request UX test command and removed bug Something isn't working labels Apr 7, 2025
@tgodzik
Copy link
Member

tgodzik commented Apr 7, 2025

We should be able to use multiple test frameworks in a single target. Maybe we generate just one in Bloop config instead of both of them? Or is the test being run separetely without Bloop?

@Gedochao
Copy link
Contributor

Gedochao commented Apr 7, 2025

We should be able to use multiple test frameworks in a single target. Maybe we generate just one in Bloop config instead of both of them? Or is the test being run separetely without Bloop?

We don't run tests with Bloop.
Rather, we have a dedicated test runner module.

@Gedochao
Copy link
Contributor

Gedochao commented Apr 7, 2025

Actually... perhaps we could just run all the test frameworks we can find, unless a framework is explicitly specified.
I now wonder why the current implementation doesn't do that already.
Let me try some stuff.

@Gedochao Gedochao linked a pull request Apr 10, 2025 that will close this issue
@Gedochao Gedochao self-assigned this Apr 10, 2025
@Gedochao Gedochao moved this from To do to In progress in Scala CLI Board Apr 10, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request test command UX
Projects
Status: In progress
Development

Successfully merging a pull request may close this issue.

3 participants