Skip to content

Print a usable reproduction command of failed tests. #1734

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
sigurdm opened this issue Jun 28, 2022 · 5 comments · Fixed by #1736
Closed

Print a usable reproduction command of failed tests. #1734

sigurdm opened this issue Jun 28, 2022 · 5 comments · Fixed by #1736

Comments

@sigurdm
Copy link
Contributor

sigurdm commented Jun 28, 2022

Currently a failing test prints a line like so:

08:28 +54 ~9 -1: test/integration.shard/command_output_test.dart: flutter run --machine uses AppRunLogger [E] 

Running that single test again from the command-line is kind of cumbersome. One has to

  • write flutter test or dart test
  • copy-paste the test filename
  • write something like -n ""<backarrow>
  • copy paste the test name.

It would be awesome if the whole command could be printed. Something like:

08:28 +54 ~9 -1: test/integration.shard/command_output_test.dart: flutter run --machine uses AppRunLogger [E] 
To run this test again: `flutter test test/integration.shard/command_output_test.dart --plain-name="flutter run --machine uses AppRunLogger"`

With escaping of tricky characters in the name string.

@jakemac53
Copy link
Contributor

#329 is another related feature, not sure if we would want both?

@sigurdm
Copy link
Contributor Author

sigurdm commented Jun 28, 2022

#329 is another related feature, not sure if we would want both?

I would personally use both of these features.

Sometimes you want to drill down into a single failed test.
Sometimes you want to see if you managed to fix all the last failures

@jakemac53
Copy link
Contributor

jakemac53 commented Jun 28, 2022

This one is also significantly easier to implement I think (although it won't be perfect, a basic implementation should work in the 99% case)

@jakemac53
Copy link
Contributor

I looked into this a bit and this is more challenging than it might appear - at least to get the line to consistently show up in a good place.

I could pretty reliably get it to show up as the first line after the failure line, something like this:

00:11 +0 -1: some test [E]  
To run this test again: `dart test test/some_test.dart -p vm --plain-name "some test"`
< failure output here>

But I don't think that is really what you want, I think you probably want it as the last line (after the failure output). The problem is some tests tend to output more text a bit after this failure event comes through, so it ends up in the middle of the output. I think that is a result of printOnFailure calls printing slightly later or something, not quite sure.

@jakemac53
Copy link
Contributor

jakemac53 commented Jun 28, 2022

Ah, so the problem appears to be that a test is marked as "complete" as soon as the first error happens. Really we want to log this after all async expectations are done as well. I figured out how to make sure it doesn't print until the very end of the test.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants