[SWT-0006] Return the thrown error from `#expect(throws:)` and `#require(throws:)`. #780

grynspan · 2024-10-22T16:59:26Z

This PR changes the signatures of the various throws: overloads of #expect and #require so that on success they return the error that was thrown rather than Void. This then allows more ergonomic inspection of the error's properties:

let error = try #require(throws: MyError.self) {
  try f()
}
#expect(error.hasWidget)
#expect(error.userName == "John Smith")

For more information, see the proposal document.

Resolves rdar://138235250.

Further PR Details

It is not possible to overload a macro or function solely by return type without the compiler reporting `Ambiguous use of 'f()'`, so we are not able to stage this change in using `@_spi(Experimental)` without breaking test code that already imports our SPI.

This change is potentially source-breaking for tests that inadvertently forward the result of these macro invocations to an enclosing scope. For example, the compiler will start emitting a warning here:

func bar(_ pfoo: UnsafePointer<Foo>) throws { ... }

withUnsafePointer(to: foo) { pfoo in // ⚠️ Result of call to 'withUnsafePointer(to:_:)' is unused
  #expect(throws: BadFooError.self) {
    try bar(pfoo)
  }
}

This warning can be suppressed by assigning the result of #expect (or of withUnsafePointer(to:_:)) to _:

func bar(_ pfoo: UnsafePointer<Foo>) throws { ... }

withUnsafePointer(to: foo) { pfoo in
  _ = #expect(throws: BadFooError.self) {
    try bar(pfoo)
  }
}

Because #expect and #require are macros, they cannot be referenced by name like functions, so you cannot assign them to variables (and then run into trouble with the types of those variables.)

Finally, this change deprecates the variants of #expect and #require that take two closures:

#expect { // ⚠️ 'expect(_:sourceLocation:performing:throws:)' is deprecated: Examine the result of '#expect(throws:)' instead.
  ...
} throws: { error in
  guard let error = error as? FoodTruckError else {
    return false
  }
  return error.napkinCount == 0
}

These variants are no longer needed because you can simply examine the result of the other variants:

let error = #expect(throws: FoodTruckError.self) {
  ...
}
#expect(error?.napkinCount == 0)

Checklist:

Code and documentation should follow the style of the Style Guide.
If public symbols are renamed or modified, DocC references should be updated.

grynspan · 2024-10-22T16:59:33Z

@swift-ci test

grynspan · 2024-10-22T21:34:29Z

@swift-ci test

grynspan · 2024-10-22T21:54:09Z

@swift-ci test

grynspan · 2024-10-23T15:48:24Z

@swift-ci test

grynspan · 2024-10-23T21:01:19Z

@swift-ci test

dabrahams · 2024-10-24T01:42:08Z

Nice!

Documentation/Proposals/NNNN-return-errors-from-expect-throws.md

Sources/TestingMacros/Support/Additions/MacroExpansionContextAdditions.swift

Sources/Testing/Support/Additions/ResultAdditions.swift

grynspan · 2024-10-24T15:42:08Z

@swift-ci test

grynspan · 2024-10-28T19:18:53Z

@swift-ci test

grynspan · 2024-10-29T13:24:22Z

@swift-ci test Linux

Documentation/Proposals/NNNN-return-errors-from-expect-throws.md

Sources/Testing/Testing.docc/AvailabilityStubs/ExpectComplexThrows.md

Sources/Testing/Testing.docc/AvailabilityStubs/RequireComplexThrows.md

Sources/TestingMacros/Support/Additions/MacroExpansionContextAdditions.swift

This PR changes the signatures of the various `throws:` overloads of `#expect` and `#require` so that on success they return the error that was thrown rather than `Void`. This then allows more ergonomic inspection of the error's properties: ```swift let error = try #require(throws: MyError.self) { try f() } #expect(error.hasWidget) #expect(error.userName == "John Smith") ``` It is not possible to overload a macro or function solely by return type without the compiler reporting `Ambiguous use of 'f()'`, so we are not able to stage this change in using `@_spi(Experimental)` without breaking test code that already imports our SPI. This change is potentially source-breaking for tests that inadvertently forward the result of these macro invocations to an enclosing scope. For example, the compiler will start emitting a warning here: ```swift func bar(_ pfoo: UnsafePointer<Foo>) throws { ... } withUnsafePointer(to: foo) { pfoo in // ⚠️ Result of call to 'withUnsafePointer(to:_:)' is unused #expect(throws: BadFooError.self) { bar(pfoo) } } ``` This warning can be suppressed by assigning the result of `#expect` (or of `withUnsafePointer(to:_:)`) to `_`: ```swift func bar(_ pfoo: UnsafePointer<Foo>) throws { ... } withUnsafePointer(to: foo) { pfoo in _ = #expect(throws: BadFooError.self) { bar(pfoo) } } ``` Because `#expect` and `#require` are macros, they cannot be referenced by name like functions, so you cannot assign them to variables (and then run into trouble with the types of those variables.)

…` only throws if you try to cast the result to `any Error` (can't instantiate `Never`), add a way to suppress our diagnostics in our own unit tests.

Documentation/Proposals/NNNN-return-errors-from-expect-throws.md

grynspan · 2024-12-13T18:36:43Z

@swift-ci test

grynspan · 2024-12-13T18:53:08Z

@swift-ci test

grynspan · 2024-12-13T20:09:06Z

@swift-ci test

…ire(throws:)`. (#780) This PR changes the signatures of the various `throws:` overloads of `#expect` and `#require` so that on success they return the error that was thrown rather than `Void`. This then allows more ergonomic inspection of the error's properties: ```swift let error = try #require(throws: MyError.self) { try f() } #expect(error.hasWidget) #expect(error.userName == "John Smith") ``` For more information, see [the proposal document](https://github.com/swiftlang/swift-testing/blob/jgrynspan/return-errors-from-expect-throws/Documentation/Proposals/0006-return-errors-from-expect-throws.md). Resolves rdar://138235250. <details> <summary>Further PR Details</summary> It is not possible to overload a macro or function solely by return type without the compiler reporting `Ambiguous use of 'f()'`, so we are not able to stage this change in using `@_spi(Experimental)` without breaking test code that already imports our SPI. This change is potentially source-breaking for tests that inadvertently forward the result of these macro invocations to an enclosing scope. For example, the compiler will start emitting a warning here: ```swift func bar(_ pfoo: UnsafePointer<Foo>) throws { ... } withUnsafePointer(to: foo) { pfoo in // ⚠️ Result of call to 'withUnsafePointer(to:_:)' is unused #expect(throws: BadFooError.self) { try bar(pfoo) } } ``` This warning can be suppressed by assigning the result of `#expect` (or of `withUnsafePointer(to:_:)`) to `_`: ```swift func bar(_ pfoo: UnsafePointer<Foo>) throws { ... } withUnsafePointer(to: foo) { pfoo in _ = #expect(throws: BadFooError.self) { try bar(pfoo) } } ``` Because `#expect` and `#require` are macros, they cannot be referenced by name like functions, so you cannot assign them to variables (and then run into trouble with the types of those variables.) Finally, this change deprecates the variants of `#expect` and `#require` that take two closures: ```swift #expect { // ⚠️ 'expect(_:sourceLocation:performing:throws:)' is deprecated: Examine the result of '#expect(throws:)' instead. ... } throws: { error in guard let error = error as? FoodTruckError else { return false } return error.napkinCount == 0 } ``` These variants are no longer needed because you can simply examine the result of the other variants: ```swift let error = #expect(throws: FoodTruckError.self) { ... } #expect(error?.napkinCount == 0) ``` </details> ### Checklist: - [x] Code and documentation should follow the style of the [Style Guide](https://github.com/apple/swift-testing/blob/main/Documentation/StyleGuide.md). - [x] If public symbols are renamed or modified, DocC references should be updated.

@stmontgomery

…`#require(throws:)`. (#857) - **Explanation**: Change `#expect(throws:)` and `#require(throws:)` to return the thrown error and deprecate the double-closure overloads. - **Scope**: 6.1 release, all tests that use these macros - **Issues**: rdar://138235250 - **Original PRs**: #780 - **Risk**: Low (some developers may get a new diagnostic about an unused return value, but this is a warning unless `-Werror`. - **Testing**: Unit test coverage. - **Reviewers**: @stmontgomery @briancroom

drseg · 2025-03-04T20:40:09Z

Apologies if I'm being completely dumb, but this seems to have gone into the current, Swift 6.0 official docs online even though as far as I can see it's not implemented until Swift 6.1...

See https://developer.apple.com/documentation/testing/expect(throws:_:sourcelocation:performing:)-1hfms for example

grynspan · 2025-03-04T20:42:22Z

This is a limitation of Apple's documentation system. See this forum thread for more discussion.

grynspan added enhancement New feature or request public-api Affects public API issue-handling Related to Issue handling within the testing library labels Oct 22, 2024

grynspan added this to the Swift 6.1 milestone Oct 22, 2024

grynspan self-assigned this Oct 22, 2024

grynspan added a commit that referenced this pull request Oct 22, 2024

Make result type of expectations optional for consistency with #780

1d3b1d4

grynspan force-pushed the jgrynspan/return-errors-from-expect-throws branch from e15c39e to 61854c2 Compare October 22, 2024 21:03

grynspan mentioned this pull request Oct 22, 2024

Support for validating the content of recorded Issues #155

Open

grynspan force-pushed the jgrynspan/return-errors-from-expect-throws branch from e58a2ce to 89c147c Compare October 23, 2024 14:47

grynspan changed the title ~~[DNM] Return the thrown error from #expect(throws:) and #require(throws:).~~ [SWT-NNNN] Return the thrown error from #expect(throws:) and #require(throws:). Oct 23, 2024

grynspan added the api-proposal API proposal PRs (documentation only) label Oct 23, 2024

grynspan marked this pull request as ready for review October 23, 2024 15:48

grynspan requested review from stmontgomery, dennisweissmann, briancroom, SeanROlszewski and suzannaratcliff as code owners October 23, 2024 15:48

grynspan commented Oct 24, 2024

View reviewed changes

Documentation/Proposals/NNNN-return-errors-from-expect-throws.md Outdated Show resolved Hide resolved

grynspan commented Oct 24, 2024

View reviewed changes

Sources/TestingMacros/Support/Additions/MacroExpansionContextAdditions.swift Show resolved Hide resolved

Sources/Testing/Support/Additions/ResultAdditions.swift Outdated Show resolved Hide resolved

grynspan force-pushed the jgrynspan/return-errors-from-expect-throws branch 2 times, most recently from 7cfe55b to 6d4fd83 Compare October 28, 2024 12:35

stmontgomery reviewed Dec 13, 2024

View reviewed changes

grynspan added 11 commits December 13, 2024 12:56

More docs

7e109e0

Deprecate double-closure variants of #expect/#require

bd37968

Suppress some warnings in tests

a9eb584

Add more test coverage, ensure that `try #require(throws: Never.self)…

3fa580a

…` only throws if you try to cast the result to `any Error` (can't instantiate `Never`), add a way to suppress our diagnostics in our own unit tests.

Add pitch document

a8c2924

Add pitch link

b1c2dbd

Address my own feedback

e727af0

Remove more externalMacro uses (oops)

82b26d9

One more

257169d

Update migration doc advice for XCTAssertThrowsError

c59ee36

stmontgomery reviewed Dec 13, 2024

View reviewed changes

Documentation/Proposals/NNNN-return-errors-from-expect-throws.md Outdated Show resolved Hide resolved

Incorporate feedback

59aee57

grynspan requested a review from stmontgomery December 13, 2024 18:36

grynspan force-pushed the jgrynspan/return-errors-from-expect-throws branch from 76fc544 to 59aee57 Compare December 13, 2024 18:52

Update proposal number

50ddd34

grynspan changed the title ~~[SWT-NNNN] Return the thrown error from #expect(throws:) and #require(throws:).~~ [SWT-0006] Return the thrown error from #expect(throws:) and #require(throws:). Dec 13, 2024

stmontgomery approved these changes Dec 13, 2024

View reviewed changes

grynspan merged commit 0b77119 into main Dec 13, 2024
3 checks passed

grynspan deleted the jgrynspan/return-errors-from-expect-throws branch December 13, 2024 20:25

grynspan mentioned this pull request Dec 13, 2024

[6.1] [SWT-0006] Return the thrown error from #expect(throws:) and #require(throws:). #857

Merged

glbrntt mentioned this pull request Dec 20, 2024

Undeprecate the various throws: expectation/requirements #873

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[SWT-0006] Return the thrown error from `#expect(throws:)` and `#require(throws:)`. #780

[SWT-0006] Return the thrown error from `#expect(throws:)` and `#require(throws:)`. #780

grynspan commented Oct 22, 2024 •

edited

Loading

grynspan commented Oct 22, 2024

grynspan commented Oct 22, 2024

grynspan commented Oct 22, 2024

grynspan commented Oct 23, 2024

grynspan commented Oct 23, 2024

dabrahams commented Oct 24, 2024

grynspan commented Oct 24, 2024

grynspan commented Oct 28, 2024

grynspan commented Oct 29, 2024

grynspan commented Dec 13, 2024

grynspan commented Dec 13, 2024

grynspan commented Dec 13, 2024

drseg commented Mar 4, 2025 •

edited

Loading

grynspan commented Mar 4, 2025

[SWT-0006] Return the thrown error from #expect(throws:) and #require(throws:). #780

[SWT-0006] Return the thrown error from #expect(throws:) and #require(throws:). #780

Conversation

grynspan commented Oct 22, 2024 • edited Loading

Checklist:

grynspan commented Oct 22, 2024

grynspan commented Oct 22, 2024

grynspan commented Oct 22, 2024

grynspan commented Oct 23, 2024

grynspan commented Oct 23, 2024

dabrahams commented Oct 24, 2024

grynspan commented Oct 24, 2024

grynspan commented Oct 28, 2024

grynspan commented Oct 29, 2024

grynspan commented Dec 13, 2024

grynspan commented Dec 13, 2024

grynspan commented Dec 13, 2024

drseg commented Mar 4, 2025 • edited Loading

grynspan commented Mar 4, 2025

[SWT-0006] Return the thrown error from `#expect(throws:)` and `#require(throws:)`. #780

[SWT-0006] Return the thrown error from `#expect(throws:)` and `#require(throws:)`. #780

grynspan commented Oct 22, 2024 •

edited

Loading

drseg commented Mar 4, 2025 •

edited

Loading