Skip to content

Repair the Windows SwiftPM build #5068

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Aug 16, 2024

Conversation

jmschonfeld
Copy link
Contributor

We currently run unit tests via SwiftPM on linux, but have not yet enabled this on Windows due to the Windows build failing. This is a large step forward towards getting the Windows build working via SwiftPM as it resolves a few main issues:

  1. Dependencies like libxml2 and curl (which depends on zlib) are found via pkgconfig on Linux, however the same functionality does not exist on Windows operating systems. Additionally, we cannot find these libraries in the toolchain like Dispatch because, unlike libdispatch, curl/libxml2 are not distributed in the toolchain (they only exist as static libraries linked into FoundationNetworking/FoundationXML respectively. Instead, on windows we require that the header include and library search paths are provided via environment variables. While this doesn't make this easy for local development, it does allow us to run these tests in Windows toolchain CI where we have already built copies of libxml2/curl for use in the toolchain, and we can provide paths into the build directory for this projects
  2. We also need to ensure that __CFInitialize properly runs. In the SwiftPM build, Foundation is statically linked into the test executable. This means that the DllMain that the toolchain build relies upon will not be invoked. Instead, we use an __attribute__((constructor)) to initialize CF when the executable is launched
  3. We disable any tests that rely on the xdgTestHelper executable. There is no way within SwiftPM to specify a dependency on the xdgTestHelper without linking its main.swift.o object file into the test executable. This happens to work on Linux due to a linker feature we've implemented on that platform, but causes symbol collisions on Windows due to the lack of this linker feature. Since there is no official support for this feature in SwiftPM packages, we must instead disable these tests.

The tests don't run successfully yet (we crash in the DateFormatter tests due to a CFTimeZone issue that I'm investigating), but with this I can at least successfully launch the tests which is a great start.

// While this works on Linux due to special linker functionality, this doesn't work on Windows and results in a collision between the two main symbols
// SwiftPM also cannot support depending on this executable (to ensure it is built) without also linking its objects into the test runner
// For those reasons, using the xdgTestHelper on Windows is currently unsupported and tests that rely on it must be skipped
throw XCTSkip("xdgTestHelper is not supported during testing on Windows")
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, this was usable when we were running the tests through CMake. I think that the issue is limited to SPM which doesn't build the libraries and tries to link the executable object library into another executable. Can we clarify this to be unsupported when running with SPM?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes this is a limitation of SwiftPM that I've brought up to some of the SwiftPM folks - I see we have a few GitHub issues tracking it. SwiftPM attempts to link the object files from the executable into the test runner, and while this happens to work on Linux on Windows the two main functions have conflicting symbols and linking fails.

Can we clarify this to be unsupported when running with SPM?

Is there something that you'd like me to add in addition to the comment that I added here? I think that comment should explain why SwiftPM cannot support this but I'm happy to add any details that you think are missing.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we update the message of the XCTSkip to call out that this is unsupported with SPM?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, added in 595a4e9

@jmschonfeld
Copy link
Contributor Author

@swift-ci please test

@jmschonfeld
Copy link
Contributor Author

@swift-ci please test Linux platform

@jmschonfeld
Copy link
Contributor Author

@swift-ci please test

@jmschonfeld
Copy link
Contributor Author

@swift-ci please test Windows platform

Copy link
Contributor

@itingliu itingliu left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking question: do we happen to have an issue tracking SwiftPM support?

@jmschonfeld
Copy link
Contributor Author

Non-blocking question: do we happen to have an issue tracking SwiftPM support?

swiftlang/swift-package-manager#6367 is tracking this issue, and I think it's effectively the same as swiftlang/swift-package-manager#7174 as well (also rdar://133768835)

@jmschonfeld jmschonfeld merged commit a39f398 into swiftlang:main Aug 16, 2024
2 of 3 checks passed
@jmschonfeld jmschonfeld deleted the windows/swiftpm-build branch August 16, 2024 20:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants