Use swift-testing stdout as test results log verbatim #1058
+11
−68
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Instead of reconstructing the swift-testing output from a combination of stdout and the JSON event stream, simply pass stdout through directly and use that as the test results log. Use the JSON event stream to drive test pass/fail/skip/issue/etc behaviour. This resolves a race condition between the JSON event stream and stdout interleaving output in an incorrect order. This approach has the drawback of no longer associating individual lines of output derived from the JSON event stream with a specific test, however this was offering limited benefit since user output wasn't captured in this way. Once every line of output can be accounted for in the JSON event stream we can add this functionality back in with user output included. Issue: swiftlang#1054
plemarquand
force-pushed
the
swift-testing-stdout-as-test-results-log
branch
from
ed0a9d8 to
e61d39b
Compare
plemarquand
requested review from
adam-fowler,
0xTim and
award999
as code owners
award999
approved these changes
Sep 9, 2024
matthewbastien
approved these changes
Sep 9, 2024
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Instead of reconstructing the swift-testing output from a combination of stdout and the JSON event stream, simply pass stdout through directly and use that as the test results log. Use the JSON event stream to drive test pass/fail/skip/issue/etc behaviour.
This resolves a race condition between the JSON event stream and stdout interleaving output in an incorrect order.
This approach has the drawback of no longer associating individual lines of output derived from the JSON event stream with a specific test, however this was offering limited benefit since user output wasn't captured in this way. Once every line of output can be accounted for in the JSON event stream we can add this functionality back in with user output included.
Issue: #1054