|
| 1 | +# Make .serialized trait API |
| 2 | + |
| 3 | +* Proposal: [SWT-NNNN](NNNN-make-serialized-trait-api.md) |
| 4 | +* Authors: [Dennis Weissmann](https://github.com/dennisweissmann) |
| 5 | +* Status: **Awaiting review** |
| 6 | +* Implementation: |
| 7 | +[apple/swift-testing#NNNNN](https://github.com/apple/swift-testing/pull/NNNNN) |
| 8 | +* Review: |
| 9 | +([pitch](https://forums.swift.org/t/pitch-make-serialized-trait-public-api/73147 |
| 10 | +)) |
| 11 | + |
| 12 | +## Introduction |
| 13 | + |
| 14 | +We propose promoting the existing `.serialized` trait to public API. This trait |
| 15 | +enables developers to designate tests or test suites to run serially, ensuring |
| 16 | +sequential execution where necessary. |
| 17 | + |
| 18 | +## Motivation |
| 19 | + |
| 20 | +The Swift Testing library defaults to parallel execution of tests, promoting |
| 21 | +efficiency and isolation. However, certain test scenarios demand strict |
| 22 | +sequential execution due to shared state or complex dependencies between tests. |
| 23 | +The `.serialized` trait provides a solution by allowing developers to enforce |
| 24 | +serial execution for specific tests or suites. |
| 25 | + |
| 26 | +While global actors ensure that only one task associated with that actor runs |
| 27 | +at any given time, thus preventing concurrent access to actor state, tasks can |
| 28 | +yield and allow other tasks to proceed, potentially interleaving execution. |
| 29 | +That means global actors do not ensure that a specific test runs entirely to |
| 30 | +completion before another begins. A testing library requires a construct that |
| 31 | +guarantees that each annotated test runs independently and completely (in its |
| 32 | +suite), one after another, without interleaving. |
| 33 | + |
| 34 | +## Proposed Solution |
| 35 | + |
| 36 | +We propose exposing the `.serialized` trait as a public API. This attribute can |
| 37 | +be applied to individual test functions or entire test suites, modifying the |
| 38 | +test execution behavior to enforce sequential execution where specified. |
| 39 | + |
| 40 | +Annotating just a single test in a suite does not enforce any serialization |
| 41 | +behavior - the testing library encourages parallelization and the bar to |
| 42 | +degrade overall performance of test execution should be high. |
| 43 | +Additionally, traits apply inwards - it would be unexpected to impact the exact |
| 44 | +conditions of a another test in a suite without applying a trait to the suite |
| 45 | +itself. |
| 46 | +Thus, this trait should only be applied to suites (to enforce serial execution |
| 47 | +of all tests inside it) or parameterized tests. If applied to just a test this |
| 48 | +trait does not have any effect. |
| 49 | + |
| 50 | +## Detailed Design |
| 51 | + |
| 52 | +The `.serialized` trait functions as an attribute that alters the execution |
| 53 | +scheduling of tests. When applied, it ensures that tests or suites annotated |
| 54 | +with `.serialized` run serially. |
| 55 | + |
| 56 | +```swift |
| 57 | +/// A type that affects whether or not a test or suite is parallelized. |
| 58 | +/// |
| 59 | +/// When added to a parameterized test function, this trait causes that test to |
| 60 | +/// run its cases serially instead of in parallel. When applied to a |
| 61 | +/// non-parameterized test function, this trait has no effect. When applied to a |
| 62 | +/// test suite, this trait causes that suite to run its contained test functions |
| 63 | +/// and sub-suites serially instead of in parallel. |
| 64 | +/// |
| 65 | +/// This trait is recursively applied: if it is applied to a suite, any |
| 66 | +/// parameterized tests or test suites contained in that suite are also |
| 67 | +/// serialized (as are any tests contained in those suites, and so on.) |
| 68 | +/// |
| 69 | +/// This trait does not affect the execution of a test relative to its peers or |
| 70 | +/// to unrelated tests. This trait has no effect if test parallelization is |
| 71 | +/// globally disabled (by, for example, passing `--no-parallel` to the |
| 72 | +/// `swift test` command.) |
| 73 | +/// |
| 74 | +/// To add this trait to a test, use ``Trait/serialized``. |
| 75 | +public struct ParallelizationTrait: TestTrait, SuiteTrait {} |
| 76 | + |
| 77 | +extension Trait where Self == ParallelizationTrait { |
| 78 | + /// A trait that serializes the test to which it is applied. |
| 79 | + /// |
| 80 | + /// ## See Also |
| 81 | + /// |
| 82 | + /// - ``ParallelizationTrait`` |
| 83 | + public static var serialized: Self { get } |
| 84 | +} |
| 85 | +``` |
| 86 | + |
| 87 | +The call site looks like this: |
| 88 | + |
| 89 | +```swift |
| 90 | +@Test(.serialized, arguments: Food.allCases) func prepare(food: Food) { |
| 91 | + // This function will be invoked serially, once per food, because it has the |
| 92 | + // .serialized trait. |
| 93 | +} |
| 94 | + |
| 95 | +@Suite(.serialized) struct FoodTruckTests { |
| 96 | + @Test(arguments: Condiment.allCases) func refill(condiment: Condiment) { |
| 97 | + // This function will be invoked serially, once per condiment, because the |
| 98 | + // containing suite has the .serialized trait. |
| 99 | + } |
| 100 | + |
| 101 | + @Test func startEngine() async throws { |
| 102 | + // This function will not run while refill(condiment:) is running. One test |
| 103 | + // must end before the other will start. |
| 104 | + } |
| 105 | +} |
| 106 | + |
| 107 | +@Suite struct FoodTruckTests { |
| 108 | + @Test(.serialized) func startEngine() async throws { |
| 109 | + // This function will not run serially - it's not a parameterized test and |
| 110 | + // the suite is not annotated with the `.serialized` trait. |
| 111 | + } |
| 112 | + |
| 113 | + @Test func prepareFood() async throws { |
| 114 | + // It doesn't matter if this test is `.serialized` or not, traits applied |
| 115 | +to other tests won't affect this test |
| 116 | + // don't impact other tests. |
| 117 | + } |
| 118 | +} |
| 119 | +``` |
| 120 | + |
| 121 | +## Source Compatibility |
| 122 | + |
| 123 | +Introducing `.serialized` as a public API does not have any impact on existing |
| 124 | +code. Tests will continue to run in parallel by default unless explicitly |
| 125 | +marked with `.serialized`. |
| 126 | + |
| 127 | +## Integration with Supporting Tools |
| 128 | + |
| 129 | +N/A. |
| 130 | + |
| 131 | +## Future Directions |
| 132 | + |
| 133 | +N/A. |
| 134 | + |
| 135 | +## Alternatives Considered |
| 136 | + |
| 137 | +Alternative approaches, such as relying solely on global actors for test |
| 138 | +isolation, were considered. However, global actors do not provide the |
| 139 | +deterministic, sequential execution required for certain testing scenarios. The |
| 140 | +`.serialized` trait offers a more explicit and flexible mechanism, ensuring |
| 141 | +that each designated test or suite runs to completion without interruption. |
| 142 | + |
| 143 | +Various more complex parallelization and serialization options were discussed |
| 144 | +and considered but ultimately disregarded in favor of this simple yet powerful |
| 145 | +implementation. |
| 146 | + |
| 147 | +## Acknowledgments |
| 148 | + |
| 149 | +Thanks to the swift-testing team and managers for their contributions! Thanks |
| 150 | +to our community for the initial feedback around this feature. |
0 commit comments