|
| 1 | +# Make .serialized trait API |
| 2 | + |
| 3 | +* Proposal: [SWT-NNNN](NNNN-make-serialized-trait-api.md) |
| 4 | +* Authors: [Dennis Weissmann](https://github.com/dennisweissmann) |
| 5 | +* Status: **Awaiting review** |
| 6 | +* Implementation: [apple/swift-testing#NNNNN](https://github.com/apple/swift-testing/pull/NNNNN) |
| 7 | +* Review: ([pitch](https://forums.swift.org/t/pitch-make-serialized-trait-public-api/73147)) |
| 8 | + |
| 9 | +## Introduction |
| 10 | + |
| 11 | +We propose promoting the existing `.serialized` trait to public API. This trait enables developers to designate tests or test suites to run serially, ensuring sequential execution where necessary. |
| 12 | + |
| 13 | +## Motivation |
| 14 | + |
| 15 | +The Swift Testing library defaults to parallel execution of tests, promoting efficiency and isolation. However, certain test scenarios demand strict sequential execution due to shared state or complex dependencies between tests. The `.serialized` trait provides a solution by allowing developers to enforce serial execution for specific tests or suites. |
| 16 | + |
| 17 | +While global actors ensure that only one task associated with that actor runs at any given time, thus preventing concurrent access to actor state, tasks can yield and allow other tasks to proceed, potentially interleaving execution. That means global actors do not ensure that a specific test runs entirely to completion before another begins. A testing library requires a construct that guarantees that each annotated test runs independently and completely (in its suite), one after another, without interleaving. |
| 18 | + |
| 19 | +## Proposed Solution |
| 20 | + |
| 21 | +We propose exposing the `.serialized` trait as a public API. This attribute can be applied to individual test functions or entire test suites, modifying the test execution behavior to enforce sequential execution where specified. |
| 22 | + |
| 23 | +Annotating just a single test in a suite does not enforce any serialization behavior - the testing library encourages parallelization and the bar to degrade overall performance of test execution should be high. |
| 24 | +Additionally, traits apply inwards - it would be unexpected to impact the exact conditions of a another test in a suite without applying a trait to the suite itself. |
| 25 | +Thus, this trait should only be applied to suites (to enforce serial execution of all tests inside it) or parameterized tests. If applied to just a test this trait does not have any effect. |
| 26 | + |
| 27 | +## Detailed Design |
| 28 | + |
| 29 | +The `.serialized` trait functions as an attribute that alters the execution scheduling of tests. When applied, it ensures that tests or suites annotated with `.serialized` run serially. |
| 30 | + |
| 31 | +```swift |
| 32 | +/// A type that affects whether or not a test or suite is parallelized. |
| 33 | +/// |
| 34 | +/// When added to a parameterized test function, this trait causes that test to |
| 35 | +/// run its cases serially instead of in parallel. When applied to a |
| 36 | +/// non-parameterized test function, this trait has no effect. When applied to a |
| 37 | +/// test suite, this trait causes that suite to run its contained test functions |
| 38 | +/// and sub-suites serially instead of in parallel. |
| 39 | +/// |
| 40 | +/// This trait is recursively applied: if it is applied to a suite, any |
| 41 | +/// parameterized tests or test suites contained in that suite are also |
| 42 | +/// serialized (as are any tests contained in those suites, and so on.) |
| 43 | +/// |
| 44 | +/// This trait does not affect the execution of a test relative to its peers or |
| 45 | +/// to unrelated tests. This trait has no effect if test parallelization is |
| 46 | +/// globally disabled (by, for example, passing `--no-parallel` to the |
| 47 | +/// `swift test` command.) |
| 48 | +/// |
| 49 | +/// To add this trait to a test, use ``Trait/serialized``. |
| 50 | +public struct ParallelizationTrait: TestTrait, SuiteTrait { |
| 51 | + public var isRecursive: Bool |
| 52 | +} |
| 53 | + |
| 54 | +extension Trait where Self == ParallelizationTrait { |
| 55 | + /// A trait that serializes the test to which it is applied. |
| 56 | + /// |
| 57 | + /// ## See Also |
| 58 | + /// |
| 59 | + /// - ``ParallelizationTrait`` |
| 60 | + public static var serialized: Self |
| 61 | +} |
| 62 | +``` |
| 63 | + |
| 64 | +The call site looks like this: |
| 65 | + |
| 66 | +```swift |
| 67 | +@Test(.serialized, arguments: Food.allCases) func prepare(food: Food) { |
| 68 | + // This function will be invoked serially, once per food, because it has the |
| 69 | + // .serialized trait. |
| 70 | +} |
| 71 | + |
| 72 | +@Suite(.serialized) struct FoodTruckTests { |
| 73 | + @Test(arguments: Condiment.allCases) func refill(condiment: Condiment) { |
| 74 | + // This function will be invoked serially, once per condiment, because the |
| 75 | + // containing suite has the .serialized trait. |
| 76 | + } |
| 77 | + |
| 78 | + @Test func startEngine() async throws { |
| 79 | + // This function will not run while refill(condiment:) is running. One test |
| 80 | + // must end before the other will start. |
| 81 | + } |
| 82 | +} |
| 83 | + |
| 84 | +@Suite struct FoodTruckTests { |
| 85 | + @Test(.serialized) func startEngine() async throws { |
| 86 | + // This function will not run serially - it's not a parameterized test and |
| 87 | + // the suite is not annotated with the `.serialized` trait. |
| 88 | + } |
| 89 | + |
| 90 | + @Test func prepareFood() async throws { |
| 91 | + // It doesn't matter if this test is `.serialized` or not, traits of other tests |
| 92 | + // don't impact other tests. |
| 93 | + } |
| 94 | +} |
| 95 | +``` |
| 96 | + |
| 97 | +## Source Compatibility |
| 98 | + |
| 99 | +Introducing `.serialized` as a public API does not have any impact on existing code. Tests will continue to run in parallel by default unless explicitly marked with `.serialized`. |
| 100 | + |
| 101 | +## Integration with Supporting Tools |
| 102 | + |
| 103 | +N/A. |
| 104 | + |
| 105 | +## Future Directions |
| 106 | + |
| 107 | +N/A. |
| 108 | + |
| 109 | +## Alternatives Considered |
| 110 | + |
| 111 | +Alternative approaches, such as relying solely on global actors for test isolation, were considered. However, global actors do not provide the deterministic, sequential execution required for certain testing scenarios. The `.serialized` trait offers a more explicit and flexible mechanism, ensuring that each designated test or suite runs to completion without interruption. |
| 112 | + |
| 113 | +Various more complex parallelization and serialization options were discussed and considered but ultimately disregarded in favor of this simple yet powerful implementation. |
| 114 | + |
| 115 | +## Acknowledgments |
| 116 | + |
| 117 | +Thanks to the swift-testing team and managers for their contributions! Thanks to our community for the initial feedback around this feature. |
0 commit comments