Add support for swift-testing #775
Conversation
plemarquand
force-pushed
the
swift-testing-support
branch
from
9266bee to
5e4ad57
Compare
plemarquand
requested review from
adam-fowler,
daveyc123,
matthewbastien and
award999
|
|
||
| return testBuildConfig; | ||
| } else { | ||
| const testBuildConfig = createSwiftTestConfiguration(folderContext, fifoPipePath, true); |
There was a problem hiding this comment.
mkfifo isn't called here. Is it cleaned up anywhere? You could use TemporaryFolder.withTemporaryFile to create a temporary file name, which will get deleted once you return found the supplied closure.
There was a problem hiding this comment.
I ended up just rming the pipe at the end of the test execution because TemporaryFolder.withTemporaryFile's API didn't fit quite right for windows pipes.
There was a problem hiding this comment.
Yeah there are cases where you exit the function before the fifo is deleted (if cancellation has been requested). Also some of those functions called might throw errors. I'd rather we did something safer with a similar style to withTemporaryFile.
// do platform specific setup of fifo and run function and then remove fifo
withTemporaryFifo(fifoPath => {
const testBuildConfig =
await LaunchConfigurations.createLaunchConfigurationForSwiftTesting(
etc
});
There was a problem hiding this comment.
Agreed—that would help encapsulate the platform-specific FIFO code too.
plemarquand
force-pushed
the
swift-testing-support
branch
from
86fc01b to
d4dd446
Compare
src/TestExplorer/TestRunner.ts
Outdated
| parsedOutputStream, | ||
| outputStream, | ||
| testBuildConfig | ||
| ); |
There was a problem hiding this comment.
Both parallel and coverage sessions don't seem to run swift testing tests
There was a problem hiding this comment.
This will come in a subsequent patch as there are some changes in SwiftPM required to support our coverage workflow. Specifically we need the --enable-code-coverage flag to be added to swift build (see swiftlang/swift-package-manager#7518).
I have the code written, I just need to test the SwiftPM update with a nightly toolchain, which I should be able to do shortly.
Support running swift-testing tests the same as XCTests. If a test run has both types, swift-testing tests will be run first followed by XCTests. First a list of XCTests and swift-testing tests to run is parsed from the test request. Test type is determined by a tag on each `vscode.TestItem[]`, either `"XCTest"` or `"swift-testing"`. swift-testing tests are launched by running the binary named <PackageName>PackageTests.swift-testing inside the build debug folder. This binary is run with the `--experimental-event-stream-output` flag which forwards test events (test started, complete, issue recorded, etc) to a named pipe. The `SwiftTestingOutputParser` watches this pipe for events as the tests are being run and translates them in to `ITestRunner` calls to record test progress in VSCode. There are different named pipe reader implementations between macOS/Linux and Windows. TODO: Coverage on swift-testing tests is not supported until swiftlang/swift-package-manager#7518 is available.
Also define a stable sort order on TestItems
plemarquand
force-pushed
the
swift-testing-support
branch
from
ccd4238 to
de1397c
Compare
|
@adam-fowler The related swift-testing changes are merged. If you're trying this out the |
|
I can tag 0.9 any time, although I've been trying to maintain a beginning-of-the-month cadence. |
|
@grynspan We have a few other things to finish before doing the next release of the extension, so you can probably hold off for a two weeks or so before doing a swift-testing release. |
Support running swift-testing tests the same as XCTests. If a test run has both types, swift-testing tests will be run first followed by XCTests.
First a list of XCTests and swift-testing tests to run is parsed from the test request. Test type is determined by a tag on each
vscode.TestItem[], either"XCTest"or"swift-testing".swift-testing tests are launched by running the binary named PackageTests.swift-testing inside the build debug folder. This binary is run with the
--experimental-event-stream-outputflag which forwards test events (test started, complete, issue recorded, etc) to a named pipe. TheSwiftTestingOutputParserwatches this pipe for events as the tests are being run and translates them in toITestRunnercalls to record test progress in VSCode.There are different named pipe reader implementations between macOS/Linux and Windows.