Skip to content

swift test prints error: fatalError twice on build failure #6605

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
MaxDesiatov opened this issue May 23, 2023 · 1 comment
Closed

swift test prints error: fatalError twice on build failure #6605

MaxDesiatov opened this issue May 23, 2023 · 1 comment
Labels
bug swift test Changes impacting `swift test` tool

Comments

@MaxDesiatov
Copy link
Contributor

MaxDesiatov commented May 23, 2023 •
edited
Loading

Description

In addition to an unhelpful error message it is printed twice when swift test fails to build a test target.

Expected behavior

An error message is printed saying that tests couldn't be run as they failed to build. The error message should be printed only once.

Actual behavior

error: fatalError message is printed twice.

Steps to reproduce

  1. swift package init --type library
  2. Open the automatically generated test case.
  3. Add this code snippet to it:
    func testExample() throws {
        let x = 0
        let y = 1 / x
        print(y)
    }
  1. Run swift test on the command line.

Swift Package Manager version/commit hash

Latest SwiftPM nightly from main

Swift & OS version (output of swift --version ; uname -a)

Swift development snapshot, macOS 13
@MaxDesiatov MaxDesiatov self-assigned this May 23, 2023
@grynspan grynspan added the swift test Changes impacting `swift test` tool label Jul 24, 2024
@MaxDesiatov MaxDesiatov removed their assignment Sep 18, 2024
@MaxDesiatov
Copy link
Contributor Author

No longer reproducible with swift-DEVELOPMENT-SNAPSHOT-2024-10-08-a.

@MaxDesiatov MaxDesiatov closed this as not planned Won't fix, can't repro, duplicate, stale Oct 25, 2024
@bkhouri bkhouri mentioned this issue Oct 25, 2024
Merged
dschaefer2 pushed a commit that referenced this issue Nov 4, 2024
@bkhouri
Test: ensure swift-[build|test] emit expected number of fatal error o…
f3e3fb6 
…n console (#8076)

Add an automated tests that ensures swift-build and swift-test emit the
expected amount of `error: fatalError` message on the console output.

### Motivation:

Issue #6605 reported `error: fatalError` was emitted twice to the
console output, and has since been fixed. We should ensure the issue
does not regress.

### Modifications:

Add an automated tests that verifies `swift-build` does not emit a
`error: fatalError` in `stdout` or `stderr` when a building a test that
does not compile.

### Result:

```
❯ swift test --filter TestGetNumberOfMatches --filter testFatalErrorDisplayedOnlyOnceWhenSingleXCTestHasFatalErrorInBuildCompilaation
Building for debugging...
[10/10] Linking SwiftPMPackageTests
Build complete! (9.88s)
Test Suite 'Selected tests' started at 2024-10-25 13:59:47.463.
Test Suite 'SwiftPMPackageTests.xctest' started at 2024-10-25 13:59:47.465.
Test Suite 'BuildCommandTests' started at 2024-10-25 13:59:47.465.
Test Case '-[CommandsTests.BuildCommandTests testFatalErrorDisplayedOnlyOnceWhenSingleXCTestHasFatalErrorInBuildCompilaation]' started.
Emitted by Test | /Users/bkhouri/Documents/git/public/swiftlang/swift-package-manager/Tests/CommandsTests/BuildCommandTests.swift:685 | INFO | GIVEN we have a Swift Package that has a fatalError building the tests
Emitted by Test | /Users/bkhouri/Documents/git/public/swiftlang/swift-package-manager/Tests/CommandsTests/BuildCommandTests.swift:687 | INFO | WHEN swift-build --build-tests is executed
Emitted by Test | /Users/bkhouri/Documents/git/public/swiftlang/swift-package-manager/Tests/CommandsTests/BuildCommandTests.swift:689 | INFO | THEN I expect a failure
Emitted by Test | /Users/bkhouri/Documents/git/public/swiftlang/swift-package-manager/Tests/CommandsTests/BuildCommandTests.swift:699 | DEBUG | stdout: "[0/1] Planning build\nBuilding for debugging...\n[0/7] /private/var/folders/1b/57vwczd16h733tf_gzsx7d8c0000gn/T/Miscellaneous_Errors_FatalErrorInSingleXCTest_TypeLibrary.JjD0VE/Miscellaneous_Errors_FatalErrorInSingleXCTest_TypeLibrary/.build/arm64-apple-macosx/debug/TypeLibraryPackageTests.derived/runner.swift\n[1/7] Write sources\n[4/7] Write swift-version-3DF09919E8911743.txt\n[6/9] Emitting module TypeLibrary\n[7/9] Compiling TypeLibrary TypeLibrary.swift\n[8/11] Emitting module TypeLibraryTests\n[9/11] Compiling TypeLibraryTests TypeLibraryTests.swift\n/private/var/folders/1b/57vwczd16h733tf_gzsx7d8c0000gn/T/Miscellaneous_Errors_FatalErrorInSingleXCTest_TypeLibrary.JjD0VE/Miscellaneous_Errors_FatalErrorInSingleXCTest_TypeLibrary/Tests/TypeLibraryTests/TypeLibraryTests.swift:10:15: error: division by zero\n 8 | func testExample() throws {\n 9 |     let x = 0\n10 |     let y = 1 / x\n   |               `- error: division by zero\n11 |     print(y)\n12 | }\n"
Emitted by Test | /Users/bkhouri/Documents/git/public/swiftlang/swift-package-manager/Tests/CommandsTests/BuildCommandTests.swift:700 | DEBUG | stderr: ""
Emitted by Test | /Users/bkhouri/Documents/git/public/swiftlang/swift-package-manager/Tests/CommandsTests/BuildCommandTests.swift:701 | DEBUG | number stdout matches: 0
Emitted by Test | /Users/bkhouri/Documents/git/public/swiftlang/swift-package-manager/Tests/CommandsTests/BuildCommandTests.swift:702 | DEBUG | number stderr matches: 0
Emitted by Test | /Users/bkhouri/Documents/git/public/swiftlang/swift-package-manager/Tests/CommandsTests/BuildCommandTests.swift:704 | INFO | AND a fatal error message is not printed to the console
Test Case '-[CommandsTests.BuildCommandTests testFatalErrorDisplayedOnlyOnceWhenSingleXCTestHasFatalErrorInBuildCompilaation]' passed (3.564 seconds).
Test Suite 'BuildCommandTests' passed at 2024-10-25 13:59:51.029.
         Executed 1 test, with 0 failures (0 unexpected) in 3.564 (3.564) seconds
Test Suite 'TestGetNumberOfMatches' started at 2024-10-25 13:59:51.029.
Test Case '-[_InternalTestSupportTests.TestGetNumberOfMatches testEmptyStringMatchesOnEmptyStringZeroTimes]' started.
Test Case '-[_InternalTestSupportTests.TestGetNumberOfMatches testEmptyStringMatchesOnEmptyStringZeroTimes]' passed (0.000 seconds).
Test Case '-[_InternalTestSupportTests.TestGetNumberOfMatches testEmptyStringMatchesOnNonEmptyMultilineStringUsingTripleDoubleQuotesZeroTimes]' started.
Test Case '-[_InternalTestSupportTests.TestGetNumberOfMatches testEmptyStringMatchesOnNonEmptyMultilineStringUsingTripleDoubleQuotesZeroTimes]' passed (0.000 seconds).
Test Case '-[_InternalTestSupportTests.TestGetNumberOfMatches testEmptyStringMatchesOnNonEmptyMultilineStringWithNeLineCharacterZeroTimes]' started.
Test Case '-[_InternalTestSupportTests.TestGetNumberOfMatches testEmptyStringMatchesOnNonEmptyMultilineStringWithNeLineCharacterZeroTimes]' passed (0.000 seconds).
Test Case '-[_InternalTestSupportTests.TestGetNumberOfMatches testEmptyStringMatchesOnNonEmptySingleLineStringZeroTimes]' started.
Test Case '-[_InternalTestSupportTests.TestGetNumberOfMatches testEmptyStringMatchesOnNonEmptySingleLineStringZeroTimes]' passed (0.000 seconds).
Test Case '-[_InternalTestSupportTests.TestGetNumberOfMatches testfatalErrorMatchesOnMultiLineWithTwoOccurencesReturnsTwo]' started.
Test Case '-[_InternalTestSupportTests.TestGetNumberOfMatches testfatalErrorMatchesOnMultiLineWithTwoOccurencesReturnsTwo]' passed (0.000 seconds).
Test Case '-[_InternalTestSupportTests.TestGetNumberOfMatches testfatalErrorWithLeadingAndTrailingNewLineMatchesOnMultiLineWithOneOccurencesReturnsOne]' started.
Test Case '-[_InternalTestSupportTests.TestGetNumberOfMatches testfatalErrorWithLeadingAndTrailingNewLineMatchesOnMultiLineWithOneOccurencesReturnsOne]' passed (0.000 seconds).
Test Case '-[_InternalTestSupportTests.TestGetNumberOfMatches testfatalErrorWithLeadingNewLineMatchesOnMultiLineWithTwoOccurencesReturnsTwo]' started.
Test Case '-[_InternalTestSupportTests.TestGetNumberOfMatches testfatalErrorWithLeadingNewLineMatchesOnMultiLineWithTwoOccurencesReturnsTwo]' passed (0.000 seconds).
Test Suite 'TestGetNumberOfMatches' passed at 2024-10-25 13:59:51.030.
         Executed 7 tests, with 0 failures (0 unexpected) in 0.001 (0.001) seconds
Test Suite 'SwiftPMPackageTests.xctest' passed at 2024-10-25 13:59:51.030.
         Executed 8 tests, with 0 failures (0 unexpected) in 3.565 (3.566) seconds
Test Suite 'Selected tests' passed at 2024-10-25 13:59:51.030.
         Executed 8 tests, with 0 failures (0 unexpected) in 3.565 (3.567) seconds
◇ Test run started.
↳ Testing Library Version: 102 (arm64e-apple-macos13.0)
✔ Test run with 0 tests passed after 0.001 seconds.```
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug swift test Changes impacting `swift test` tool
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@MaxDesiatov @grynspan