Skip to content

Integration with swift-testing #7047

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 15, 2023
Merged

Conversation

grynspan
Copy link
Contributor

@grynspan grynspan commented Oct 31, 2023

This PR sets up initial integration with swift-testing. It plumbs through the concept of "testing library" from SwiftTestTool down to BuildOperationBuildSystemDelegateHandler.

Developers can opt into support for swift-testing by passing --enable-experimental-swift-testing. This causes the package manager to perform a separate build of applicable test targets using swift-testing instead of XCTest. If XCTest remains enabled (i.e. by not also explicitly passing --disable-xctest), then tests from both libraries can be run.

At build time, the package manager will now (when opted in) synthesize a main function for the binary that calls a known entry point function in swift-testing. On Darwin, the build flow for XCTest-based tests differs from that on Linux (corelibs-xctest); swift-testing's build flow is very similar to the corelibs-xctest flow and uses most of the same circuitry. A future area of exploration for us may be to adopt this flow even when using XCTest on Darwin, but it's not in the scope of our work right now.

swift-testing is responsible for interpreting any additional arguments passed to it such as --filter; this support has not been implemented yet and will be built up over time.

Example command to trigger running swift-testing's self-hosted tests (prior to an updated toolchain):

swift run swift-test --enable-experimental-swift-testing --disable-xctest --package-path /PATH/TO/swift-testing

Resolves #7027.
Resolves rdar://114997435.

@grynspan
Copy link
Contributor Author

@swift-ci please test

@grynspan
Copy link
Contributor Author

@swift-ci please test

@grynspan
Copy link
Contributor Author

@swift-ci please test Linux

@grynspan
Copy link
Contributor Author

@swift-ci please test

2 similar comments
@grynspan
Copy link
Contributor Author

grynspan commented Nov 1, 2023

@swift-ci please test

@grynspan
Copy link
Contributor Author

grynspan commented Nov 1, 2023

@swift-ci please test

@grynspan
Copy link
Contributor Author

grynspan commented Nov 1, 2023

@swift-ci please test Windows

@grynspan grynspan force-pushed the jgrynspan/swift-testing-integration branch from 170b023 to 2fe0bbb Compare November 1, 2023 18:40
@grynspan
Copy link
Contributor Author

grynspan commented Nov 1, 2023

@swift-ci please test

@neonichu
Copy link
Contributor

neonichu commented Nov 1, 2023

@swift-ci please test windows

@grynspan
Copy link
Contributor Author

grynspan commented Nov 1, 2023

@swift-ci please test

@grynspan
Copy link
Contributor Author

grynspan commented Nov 1, 2023

@swift-ci please test Windows

@grynspan
Copy link
Contributor Author

grynspan commented Nov 1, 2023

@swift-ci please test

@grynspan
Copy link
Contributor Author

grynspan commented Nov 1, 2023

@swift-ci please test Windows

@grynspan
Copy link
Contributor Author

grynspan commented Nov 2, 2023

@swift-ci please test

@grynspan
Copy link
Contributor Author

grynspan commented Nov 2, 2023

@swift-ci please test Windows

1 similar comment
@grynspan
Copy link
Contributor Author

grynspan commented Nov 2, 2023

@swift-ci please test Windows

@grynspan grynspan marked this pull request as ready for review November 2, 2023 22:12
@grynspan grynspan requested a review from MaxDesiatov as a code owner November 2, 2023 22:12
@grynspan
Copy link
Contributor Author

grynspan commented Nov 2, 2023

@swift-ci please test

@grynspan
Copy link
Contributor Author

grynspan commented Nov 2, 2023

@swift-ci please test Windows

3 similar comments
@grynspan
Copy link
Contributor Author

grynspan commented Nov 3, 2023

@swift-ci please test Windows

@neonichu
Copy link
Contributor

neonichu commented Nov 3, 2023

@swift-ci please test Windows

@grynspan
Copy link
Contributor Author

grynspan commented Nov 3, 2023

@swift-ci please test Windows

@grynspan grynspan force-pushed the jgrynspan/swift-testing-integration branch from ccda01c to b480d3c Compare November 7, 2023 18:16
@grynspan grynspan requested a review from MaxDesiatov November 7, 2023 18:16
@grynspan
Copy link
Contributor Author

grynspan commented Nov 7, 2023

@swift-ci please test

@grynspan
Copy link
Contributor Author

grynspan commented Nov 7, 2023

@swift-ci please test Windows

Copy link
Contributor

@MaxDesiatov MaxDesiatov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@grynspan
Copy link
Contributor Author

grynspan commented Nov 8, 2023

@swift-ci please test

@MaxDesiatov
Copy link
Contributor

@swift-ci test windows

@grynspan grynspan force-pushed the jgrynspan/swift-testing-integration branch from f0e2a05 to 014d8be Compare November 13, 2023 16:51
@grynspan
Copy link
Contributor Author

@swift-ci please test

@grynspan
Copy link
Contributor Author

@swift-ci please test Windows

@grynspan
Copy link
Contributor Author

@swift-ci please test

@grynspan
Copy link
Contributor Author

@swift-ci please test Windows

@grynspan grynspan force-pushed the jgrynspan/swift-testing-integration branch from 014d8be to 07fdd30 Compare November 14, 2023 17:15
This PR sets up initial integration with swift-testing. It plumbs through the concept of "testing library" from `SwiftTestTool` down to `BuildOperationBuildSystemDelegateHandler`.

Developers can opt into support for swift-testing by passing `--enable-experimental-swift-testing`. This causes the package manager to perform a _separate_ build of applicable test targets using swift-testing instead of XCTest. If XCTest remains enabled (i.e. by not also explicitly passing `--disable-xctest`), then tests from both libraries can be run.

At build time, the package manager will now (when opted in) synthesize a main function for the binary that calls a known entry point function in swift-testing. On Darwin, the build flow for XCTest-based tests differs from that on Linux (corelibs-xctest); swift-testing's build flow is very similar to the corelibs-xctest flow and uses most of the same circuitry. A future area of exploration for us may be to adopt this flow even when using XCTest on Darwin, but it's not in the scope of our work right now.

swift-testing is responsible for interpreting any additional arguments passed to it such as `--filter`; this support has not been implemented yet and will be built up over time.
@grynspan grynspan force-pushed the jgrynspan/swift-testing-integration branch from 07fdd30 to 00f0359 Compare November 14, 2023 19:51
@grynspan
Copy link
Contributor Author

@swift-ci please test

@grynspan
Copy link
Contributor Author

@swift-ci please test Windows

@grynspan
Copy link
Contributor Author

@swift-ci please test windows

@grynspan
Copy link
Contributor Author

@swift-ci please test macOS

1 similar comment
@grynspan
Copy link
Contributor Author

@swift-ci please test macOS

@grynspan grynspan merged commit 3fb4d81 into main Nov 15, 2023
@grynspan grynspan deleted the jgrynspan/swift-testing-integration branch November 15, 2023 20:48
grynspan added a commit to swiftlang/swift-testing that referenced this pull request Nov 21, 2023
We now have functional support for swift-testing in Swift Package Manager (as of swiftlang/swift-package-manager#7047). We should update our documentation:

- Point developers to the main branch instead of 5.10 again;
- Show developers how to invoke `swift test` in a way that triggers swift-testing; and
- Show developers how to use `XCTestScaffold` _only_ when using an older toolchain (with the appropriate caveats and nuances emphasized.)

Resolves #126.
grynspan added a commit to swiftlang/swift-testing that referenced this pull request Nov 27, 2023
* Update Getting Started docs to use Swift main branch features

We now have functional support for swift-testing in Swift Package Manager (as of swiftlang/swift-package-manager#7047). We should update our documentation:

- Point developers to the main branch instead of 5.10 again;
- Show developers how to invoke `swift test` in a way that triggers swift-testing; and
- Show developers how to use `XCTestScaffold` _only_ when using an older toolchain (with the appropriate caveats and nuances emphasized.)

Resolves #126.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Feature Request: support swift test --filter with swift-testing package
5 participants