Mute unit test discovery failure when test files are not syntactically correct

[scottbilas]

Environment data

VS Code version: 1.19.3
Python Extension version: 2018.1.0
Python Version: 3.6.4
OS and version: Windows 10 x64 Pro v1709

Actual behavior

I tend to save often, not just to back up work, but also to get updates from the background linter - I'm watching for the squiggly lines under text that tell me what obvious thing I broke.

Frequently, I happen to save at a time when the script won't even compile. This in turn can trigger the "[Error] There were some errors in discovering unit tests" banner, which is not only distracting, but also obscures nearly all of the tabs of my open files. I can hit ESC to get rid of it, but it comes back so often that I'm developing a weird twitch.

Expected behavior

It would be super useful to have a way to mute the banner in my settings. I don't even think it's all that useful to me anyway, given that I can see any failures during discovery in the same output window I'll be paying attention to when I run the tests.

Perhaps a "python.unitTest.suppressDiscoveringTestsWarning", similar in spirit to the suppression options I see for gitlens, csharp and lldb. Bonus points for adding a "don't show this again" option to the error banner.

(Is there already a way to do this? I've poked around and was unable to find anything.)

Steps to reproduce:

1. Create a Python project with unit tests, and enable tests (I happen to be using "python.unitTest.pyTestEnabled": true in this workspace.
2. Edit a test file, or an import of a test file, and introduce a syntax error
3. Ctrl-S
4. The UI error banner will appear and won't go away until ESC or click "Close"
5. Repeat again and again
6. Develop twitch

[DonJayamanne]

@brettcannon

• What do you think about this setting:
  "python.unitTest.suppressDiscoveringTestsWarning"
  Might want to look at a related issue Be able to disable automatic test discovery on save #1037