Skip to content

[Build] Add support for building with the integrated Swift driver #2736

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
May 13, 2020

Conversation

DougGregor
Copy link
Member

@DougGregor DougGregor commented May 8, 2020

Add bootstrap and build support for building the new Swift driver
and importing it (as a library) into SwiftPM. Then introduce the new
SwiftPM flag --use-integrated-swift-driver to enable use of the
integrated Swift driver rather than shelling out to a separate
Swift driver binary. The jobs created by the integrated Swift driver
will be merged into SwiftPM's build graph.

@davidungar
Copy link
Contributor

Cool!

@DougGregor DougGregor changed the title [Build] Add support for building and importing the new Swift driver [Build] Add support for building with the integrated Swift driver May 10, 2020
def add_rpath_for_cmake_build(args, rpath):
"Adds the given rpath to the CMake-built swift-build"
swift_build = os.path.join(args.bootstrap_dir, "bin/swift-build")
add_rpath_cmd = ["install_name_tool", "-add_rpath", rpath, swift_build]
Copy link
Contributor

@abertelrud abertelrud May 11, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be possible to do this with -rpath arguments to the linker instead? I realize that the diff in this PR is in line with the existing approach of using install_name_tool, but I've been investigating some intermittent failures that can occur when there isn't enough space in the Mach-O header for the additional load commands. The approach I was going to use was to switch to passing -rpath at link time, which seems cleaner. Alternatively, we would need to make sure that the link command leaves enough space by using -headerpad, but that's a bit messier since we'd have to guess at a number large enough to make sure there's enough space, and it would leave an unnecessary hole in the binary.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be possible, yeah; we just need to figure out how to drive CMake to do that. That should be a separate pull request, but I could certainly see the argument that we should implement that improvement before merging here, because adding a bunch more rpaths in this manner is likely to push us over the limit.

@neonichu
Copy link
Contributor

@swift-ci please smoke test

@DougGregor
Copy link
Member Author

@neonichu I should have called it out before, but this PR depends on swiftlang/swift#31664 and some Jenkins updates @shahmishal is doing. I expect it to fail bootstrap.

@DougGregor
Copy link
Member Author

Yeah, the tests failed with the expected:

 CMake Error: The source directory "/Users/buildnode/jenkins/workspace/swift-package-manager-PR-osx-smoke-test/branch-master/yams" does not exist.

@shahmishal
Copy link
Member

swiftlang/swift#31664
@swift-ci smoke test

@DougGregor
Copy link
Member Author

@swift-ci smoke test

@DougGregor
Copy link
Member Author

@swift-ci test Linux

@DougGregor
Copy link
Member Author

Ahh, a real failure! Excellent:

18:58:07 CMake Error at CMakeLists.txt:18 (find_package):
18:58:07   Could not find a package configuration file provided by "Foundation" with
18:58:07   any of the following names:
18:58:07 
18:58:07     FoundationConfig.cmake
18:58:07     foundation-config.cmake
18:58:07 
18:58:07   Add the installation prefix of "Foundation" to CMAKE_PREFIX_PATH or set
18:58:07   "Foundation_DIR" to a directory containing one of the above files.  If
18:58:07   "Foundation" provides a separate development package or SDK, be sure it has
18:58:07   been installed.

Add bootstrap and build support for building the new Swift driver
and importing it (as a library) into SwiftPM. This is an incomplete
stepping stone toward integrating the new driver.
Add missing dependencies, rpaths, etc. to get SwiftPM bootstrapping
with the new Swift driver. There is no actual integration of
SwiftDriver beyond the import.
Add the command-line flag `--use-integrated-swift-driver` to use
the integrated Swift driver to create jobs rather than shelling
out to the driver executable.
…criptions

Unique command names allow us to successfully bootstrap using the integrated
Swift driver. Using the Swift driver's job kind to provide more detailed
descriptions for the various shell commands generated by the Swift driver,
improving the output.
The Swift driver's toolchain and SwiftPM's toolchain might have slightly
different paths in them. For now, map to SwiftPM's toolchain. There
should be a more direct way to rectify these.
@DougGregor DougGregor force-pushed the integrate-swift-driver-build branch from a501d0f to c818cdb Compare May 13, 2020 00:22
@DougGregor
Copy link
Member Author

@swift-ci test

@DougGregor
Copy link
Member Author

@swift-ci smoke test

@shahmishal
Copy link
Member

@swift-ci test

@shahmishal
Copy link
Member

@swift-ci smoke test

@DougGregor DougGregor merged commit 8cfdbab into swiftlang:master May 13, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants