Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add attachments #714

Closed
Arideno opened this issue Sep 19, 2024 · 1 comment · Fixed by #770 or #796
Closed

Add attachments #714

Arideno opened this issue Sep 19, 2024 · 1 comment · Fixed by #770 or #796
Labels
attachments/activities 🖇️ Work related to attachments and/or activities enhancement New feature or request public-api Affects public API
Milestone

Comments

@Arideno
Copy link

Arideno commented Sep 19, 2024

Description

The XCTest framework has an attachment feature, which is beneficial for snapshot tests to visualize the differences. It would be advantageous if this feature were incorporated into SwiftTesting as well.

@Arideno Arideno added the enhancement New feature or request label Sep 19, 2024
@grynspan
Copy link
Contributor

Oddly timely as I'm working on a proposal for this right now!

@grynspan grynspan added the public-api Affects public API label Sep 25, 2024
grynspan added a commit that referenced this issue Oct 17, 2024
…S_SPAWNING`.

This PR separates out our process-spawning code to be guarded by
`SWT_NO_PROCESS_SPAWNING` instead of `SWT_NO_EXIT_TESTS`. We do this so that we
can potentially use process spawning on platforms where exit tests are not
supported for some other reason (such as the iOS/Android sandboxes) but process
spawning is still internally possible.

There are a few use cases we have for spawning processes that don't involve exit
tests:

- Calling out to `tar` to compress attachments (see #714)
- Running non-Swift scripts in their interpreters (see #478)
- Multi-process parallelism (the XCTest model)

I took the opportunity to clean up WaitFor.swift a bit and rearrange code so
that the "new platform, dunno who lives here" case should compile (although not
function) out-of-the-box.
@grynspan grynspan added this to the Swift 6.1 milestone Oct 17, 2024
grynspan added a commit that referenced this issue Oct 17, 2024
…S_SPAWNING`. (#769)

This PR separates out our process-spawning code to be guarded by
`SWT_NO_PROCESS_SPAWNING` instead of `SWT_NO_EXIT_TESTS`. We do this so
that we can potentially use process spawning on platforms where exit
tests are not supported for some other reason (such as the iOS/Android
sandboxes) but process spawning is still internally possible.

There are a few use cases we have for spawning processes that don't
involve exit tests:

- Calling out to `tar` to compress attachments (see #714)
- Running non-Swift scripts in their interpreters (see #478)
- Multi-process parallelism (the XCTest model)

I took the opportunity to clean up WaitFor.swift a bit and rearrange
code so that the "new platform, dunno who lives here" case should
compile (although not function) out-of-the-box.

### Checklist:

- [x] Code and documentation should follow the style of the [Style
Guide](https://github.com/apple/swift-testing/blob/main/Documentation/StyleGuide.md).
- [x] If public symbols are renamed or modified, DocC references should
be updated.
grynspan added a commit that referenced this issue Oct 17, 2024
This PR introduces a new experimental feature, attachments. With this feature,
you can "attach" values that conform to a new `Test.Attachable` protocol to a
test. With the right command-line incantation (TBD), Swift Testing will
automatically write attachments to disk for you.

> ![NOTE]
> This PR does not teach Xcode or VS Code how to handle attachments produced by
> Swift Testing, nor does it add the necessary command-line arguments to the
> `swift test` command-line tool.

This PR is one of a series that I'll be posting to build out this feature.

As always, keep in mind that symbols marked `@_spi(Experimental)` are subject to
change or removal without notice.

Resolves #714.
Resolves rdar://88648735.
@grynspan grynspan mentioned this issue Oct 17, 2024
2 tasks
grynspan added a commit that referenced this issue Oct 17, 2024
This PR introduces a new experimental feature, attachments. With this feature,
you can "attach" values that conform to a new `Test.Attachable` protocol to a
test. With the right command-line incantation (TBD), Swift Testing will
automatically write attachments to disk for you.

> ![NOTE]
> This PR does not teach Xcode or VS Code how to handle attachments produced by
> Swift Testing, nor does it add the necessary command-line arguments to the
> `swift test` command-line tool.

This PR is one of a series that I'll be posting to build out this feature.

As always, keep in mind that symbols marked `@_spi(Experimental)` are subject to
change or removal without notice.

Resolves #714.
Resolves rdar://88648735.
grynspan added a commit that referenced this issue Oct 22, 2024
This PR introduces a new experimental feature, attachments. With this feature,
you can "attach" values that conform to a new `Test.Attachable` protocol to a
test. With the right command-line incantation (TBD), Swift Testing will
automatically write attachments to disk for you.

> ![NOTE]
> This PR does not teach Xcode or VS Code how to handle attachments produced by
> Swift Testing, nor does it add the necessary command-line arguments to the
> `swift test` command-line tool.

This PR is one of a series that I'll be posting to build out this feature.

As always, keep in mind that symbols marked `@_spi(Experimental)` are subject to
change or removal without notice.

Resolves #714.
Resolves rdar://88648735.
grynspan added a commit that referenced this issue Oct 23, 2024
This PR introduces a new experimental feature, attachments. With this feature,
you can "attach" values that conform to a new `Test.Attachable` protocol to a
test. With the right command-line incantation (TBD), Swift Testing will
automatically write attachments to disk for you.

> ![NOTE]
> This PR does not teach Xcode or VS Code how to handle attachments produced by
> Swift Testing, nor does it add the necessary command-line arguments to the
> `swift test` command-line tool.

This PR is one of a series that I'll be posting to build out this feature.

As always, keep in mind that symbols marked `@_spi(Experimental)` are subject to
change or removal without notice.

Resolves #714.
Resolves rdar://88648735.
grynspan added a commit that referenced this issue Oct 25, 2024
This PR introduces a new experimental feature, attachments. With this feature,
you can "attach" values that conform to a new `Test.Attachable` protocol to a
test. With the right command-line incantation (TBD), Swift Testing will
automatically write attachments to disk for you.

> ![NOTE]
> This PR does not teach Xcode or VS Code how to handle attachments produced by
> Swift Testing, nor does it add the necessary command-line arguments to the
> `swift test` command-line tool.

This PR is one of a series that I'll be posting to build out this feature.

As always, keep in mind that symbols marked `@_spi(Experimental)` are subject to
change or removal without notice.

Resolves #714.
Resolves rdar://88648735.
grynspan added a commit that referenced this issue Oct 28, 2024
This PR introduces a new experimental feature, attachments. With this feature,
you can "attach" values that conform to a new `Test.Attachable` protocol to a
test. With the right command-line incantation (TBD), Swift Testing will
automatically write attachments to disk for you.

> ![NOTE]
> This PR does not teach Xcode or VS Code how to handle attachments produced by
> Swift Testing, nor does it add the necessary command-line arguments to the
> `swift test` command-line tool.

This PR is one of a series that I'll be posting to build out this feature.

As always, keep in mind that symbols marked `@_spi(Experimental)` are subject to
change or removal without notice.

Resolves #714.
Resolves rdar://88648735.
grynspan added a commit that referenced this issue Oct 30, 2024
This PR introduces a new experimental feature, attachments. With this feature,
you can "attach" values that conform to a new `Test.Attachable` protocol to a
test. With the right command-line incantation (TBD), Swift Testing will
automatically write attachments to disk for you.

> ![NOTE]
> This PR does not teach Xcode or VS Code how to handle attachments produced by
> Swift Testing, nor does it add the necessary command-line arguments to the
> `swift test` command-line tool.

This PR is one of a series that I'll be posting to build out this feature.

As always, keep in mind that symbols marked `@_spi(Experimental)` are subject to
change or removal without notice.

Resolves #714.
Resolves rdar://88648735.
@grynspan grynspan mentioned this issue Nov 2, 2024
2 tasks
@grynspan grynspan reopened this Nov 2, 2024
grynspan added a commit that referenced this issue Nov 3, 2024
This PR introduces a new experimental feature, attachments. With this feature,
you can "attach" values that conform to a new `Test.Attachable` protocol to a
test. With the right command-line incantation (TBD), Swift Testing will
automatically write attachments to disk for you.

> ![NOTE]
> This PR does not teach Xcode or VS Code how to handle attachments produced by
> Swift Testing, nor does it add the necessary command-line arguments to the
> `swift test` command-line tool.

This PR is one of a series that I'll be posting to build out this feature.

As always, keep in mind that symbols marked `@_spi(Experimental)` are subject to
change or removal without notice.

Resolves #714.
Resolves rdar://88648735.
@grynspan grynspan added the attachments/activities 🖇️ Work related to attachments and/or activities label Jan 20, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
attachments/activities 🖇️ Work related to attachments and/or activities enhancement New feature or request public-api Affects public API
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants