Skip to content

bootstrap: using a toolchain that includes 'rust-src', ./x check leaves a dirty working tree, fails to build #141991

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
lambdageek opened this issue Jun 3, 2025 · 3 comments · Fixed by #142000
Labels
C-bug Category: This is a bug. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

Comments

@lambdageek
Copy link
Contributor

lambdageek commented Jun 3, 2025

Summary

When building a recent nightly (after #119899) using a beta toolchain that includes the rust-src component, ./x check leaves a dirty tree (git status shows lots of modified files) and the check build fails.

Command used

git clone https://github.com/rust-lang/rust .
git checkout 5d707b07e42766c080c5012869c9988a18dcbb83

rustup install 1.88.0-beta.4
rustup component add --toolchain 1.88.0-beta.4 rust-src

./configure --set build.cargo=${HOME}/.rustup/toolchains/1.88.0-beta.4-x86_64-unknown-linux-gnu/bin/cargo --set build.rustc=${HOME}/.rustup/toolchains/1.88.0-beta.4-x86_64-unknown-linux-gnu/bin/rustc --set rust.channel=nightly --set profile=dist --set llvm.download-ci-llvm=true --set rust.download-rustc=false

./x check

Expected behaviour

./x check succeeds and git status shows a clean working tree

Actual behaviour

./x check fails. The exact failure differs based on platform and configure options, but this is one example:

Building compiler artifacts (stage0 -> stage1, x86_64-unknown-linux-gnu)
...
  Compiling rustc_proc_macro v0.0.0 (/home/aleksey/hacking/rust-boot/rust/compiler/rustc_proc_macro)
   Compiling ruzstd v0.7.3
   Compiling unicode-script v0.5.7
   Compiling libloading v0.8.7
   Compiling punycode v0.4.1
   Compiling rustc-demangle v0.1.24
   Compiling fallible-iterator v0.3.0
   Compiling regex-syntax v0.6.29
   Compiling rustc_llvm v0.0.0 (/home/aleksey/hacking/rust-boot/rust/compiler/rustc_llvm)
   Compiling lazy_static v1.5.0
error: prefer `FxHashMap` over `HashMap`, it has better performance
  --> compiler/rustc_proc_macro/../../library/proc_macro/src/bridge/fxhash.rs:12:35
   |
12 | ...hMap<K, V> = HashMap<K, V, BuildHasherDefault<FxHasher>>;
   |                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
   |
   = note: a `use rustc_data_structures::fx::FxHashMap` may be necessary
   = note: `-D rustc::default-hash-types` implied by `-D warnings`
   = help: to override `-D warnings` add `#[allow(rustc::default_hash_types)]`

error: could not compile `rustc_proc_macro` (lib) due to 1 previous error
warning: build failed, waiting for other jobs to finish...
Build completed unsuccessfully in 0:01:20

And git status shows many modified files

$ git status
On branch aleksey-wip1
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
  (commit or discard the untracked or modified content in submodules)
	modified:   library/Cargo.lock
	modified:   library/alloc/Cargo.toml
	modified:   library/alloc/src/collections/btree/map.rs
	modified:   library/alloc/src/collections/btree/map/tests.rs
	modified:   library/alloc/src/collections/btree/set.rs
	modified:   library/alloc/src/collections/btree/set/tests.rs
	modified:   library/alloc/src/collections/linked_list.rs
	modified:   library/alloc/src/collections/vec_deque/mod.rs
	modified:   library/alloc/src/raw_vec/mod.rs
	modified:   library/alloc/src/slice.rs
	modified:   library/alloc/src/str.rs
	modified:   library/alloc/src/string.rs
	modified:   library/alloc/src/vec/mod.rs
	modified:   library/alloctests/benches/btree/map.rs
	modified:   library/alloctests/benches/btree/set.rs
	modified:   library/alloctests/tests/autotraits.rs
	modified:   library/alloctests/tests/fmt.rs
	modified:   library/core/Cargo.toml
	modified:   library/core/src/alloc/layout.rs
	modified:   library/core/src/array/mod.rs
	modified:   library/core/src/ascii/ascii_char.rs
	modified:   library/core/src/cell/lazy.rs
	modified:   library/core/src/char/convert.rs
	modified:   library/core/src/char/methods.rs
	modified:   library/core/src/clone.rs
	modified:   library/core/src/convert/mod.rs
	modified:   library/core/src/ffi/c_str.rs
	modified:   library/core/src/ffi/mod.rs
	modified:   library/core/src/ffi/primitives.rs
	modified:   library/core/src/ffi/va_list.rs
	modified:   library/core/src/fmt/float.rs
	modified:   library/core/src/fmt/mod.rs
	modified:   library/core/src/fmt/num.rs
	modified:   library/core/src/fmt/rt.rs
	modified:   library/core/src/future/async_drop.rs
	modified:   library/core/src/future/mod.rs
	modified:   library/core/src/hint.rs
	modified:   library/core/src/intrinsics/mod.rs
	modified:   library/core/src/intrinsics/simd.rs
	modified:   library/core/src/iter/traits/collect.rs
	modified:   library/core/src/iter/traits/iterator.rs
	modified:   library/core/src/lib.rs
	modified:   library/core/src/macros/mod.rs
	modified:   library/core/src/marker.rs
	modified:   library/core/src/marker/variance.rs
	modified:   library/core/src/mem/maybe_uninit.rs
	modified:   library/core/src/mem/mod.rs
	modified:   library/core/src/net/ip_addr.rs
	modified:   library/core/src/num/dec2flt/float.rs
	modified:   library/core/src/num/dec2flt/mod.rs
	modified:   library/core/src/num/f128.rs
	modified:   library/core/src/num/f16.rs
	modified:   library/core/src/num/f32.rs
	modified:   library/core/src/num/f64.rs
	modified:   library/core/src/num/flt2dec/decoder.rs
	modified:   library/core/src/num/int_macros.rs
	modified:   library/core/src/num/mod.rs
	modified:   library/core/src/num/nonzero.rs
	modified:   library/core/src/num/uint_macros.rs
	modified:   library/core/src/ops/arith.rs
	modified:   library/core/src/ops/function.rs
	modified:   library/core/src/ops/index.rs
	modified:   library/core/src/ops/index_range.rs
	modified:   library/core/src/ops/try_trait.rs
	modified:   library/core/src/option.rs
	modified:   library/core/src/panicking.rs
	modified:   library/core/src/pin.rs
	modified:   library/core/src/pin/unsafe_pinned.rs
	modified:   library/core/src/prelude/v1.rs
	modified:   library/core/src/primitive_docs.rs
	modified:   library/core/src/ptr/alignment.rs
	modified:   library/core/src/ptr/const_ptr.rs
	modified:   library/core/src/ptr/mod.rs
	modified:   library/core/src/ptr/mut_ptr.rs
	modified:   library/core/src/ptr/non_null.rs
	modified:   library/core/src/result.rs
	modified:   library/core/src/slice/index.rs
	modified:   library/core/src/slice/mod.rs
	modified:   library/core/src/slice/raw.rs
	modified:   library/core/src/slice/sort/select.rs
	modified:   library/core/src/slice/sort/stable/mod.rs
	modified:   library/core/src/slice/sort/unstable/mod.rs
	modified:   library/core/src/slice/sort/unstable/quicksort.rs
	modified:   library/core/src/str/converts.rs
	modified:   library/core/src/str/mod.rs
	modified:   library/core/src/str/traits.rs
	modified:   library/core/src/sync/atomic.rs
	modified:   library/core/src/time.rs
	modified:   library/core/src/ub_checks.rs
	modified:   library/coretests/Cargo.toml
	modified:   library/coretests/tests/array.rs
	modified:   library/coretests/tests/ffi/cstr.rs
	modified:   library/coretests/tests/lib.rs
	modified:   library/coretests/tests/macros.rs
	modified:   library/coretests/tests/num/dec2flt/decimal.rs
	modified:   library/coretests/tests/num/dec2flt/float.rs
	modified:   library/coretests/tests/num/dec2flt/lemire.rs
	modified:   library/coretests/tests/num/dec2flt/mod.rs
	modified:   library/coretests/tests/num/dec2flt/parse.rs
	modified:   library/coretests/tests/num/flt2dec/mod.rs
	modified:   library/coretests/tests/num/flt2dec/random.rs
	modified:   library/coretests/tests/num/flt2dec/strategy/dragon.rs
	modified:   library/coretests/tests/num/flt2dec/strategy/grisu.rs
	modified:   library/coretests/tests/num/mod.rs
	modified:   library/coretests/tests/pin_macro.rs
	modified:   library/coretests/tests/time.rs
	modified:   library/panic_abort/Cargo.toml
	modified:   library/panic_abort/src/lib.rs
	modified:   library/panic_unwind/src/hermit.rs
	modified:   library/proc_macro/Cargo.toml
	modified:   library/proc_macro/src/bridge/arena.rs
	modified:   library/proc_macro/src/lib.rs
	modified:   library/std/Cargo.toml
	modified:   library/std/src/collections/hash/map.rs
	modified:   library/std/src/collections/hash/set.rs
	modified:   library/std/src/ffi/mod.rs
	modified:   library/std/src/ffi/os_str.rs
	modified:   library/std/src/fs.rs
	modified:   library/std/src/fs/tests.rs
	modified:   library/std/src/io/error.rs
	modified:   library/std/src/io/mod.rs
	modified:   library/std/src/io/pipe.rs
	modified:   library/std/src/io/tests.rs
	modified:   library/std/src/lib.rs
	modified:   library/std/src/os/net/linux_ext/addr.rs
	modified:   library/std/src/os/net/linux_ext/socket.rs
	modified:   library/std/src/os/net/linux_ext/tcp.rs
	modified:   library/std/src/os/unix/net/stream.rs
	modified:   library/std/src/os/unix/process.rs
	modified:   library/std/src/os/wasi/fs.rs
	modified:   library/std/src/os/windows/process.rs
	modified:   library/std/src/panic.rs
	modified:   library/std/src/panicking.rs
	modified:   library/std/src/path.rs
	modified:   library/std/src/prelude/v1.rs
	modified:   library/std/src/process.rs
	modified:   library/std/src/rt.rs
	modified:   library/std/src/sync/lazy_lock.rs
	modified:   library/std/src/sync/mpmc/list.rs
	modified:   library/std/src/sync/once_lock.rs
	modified:   library/std/src/sync/poison/mutex.rs
	modified:   library/std/src/sync/poison/rwlock.rs
	modified:   library/std/src/sync/reentrant_lock.rs
	modified:   library/std/src/sys/cmath.rs
	modified:   library/std/src/sys/fs/unix.rs
	modified:   library/std/src/sys/fs/windows.rs
	modified:   library/std/src/sys/net/connection/uefi/mod.rs
	modified:   library/std/src/sys/os_str/wtf8.rs
	modified:   library/std/src/sys/pal/hermit/mod.rs
	modified:   library/std/src/sys/pal/sgx/mod.rs
	modified:   library/std/src/sys/pal/uefi/helpers.rs
	modified:   library/std/src/sys/pal/uefi/mod.rs
	modified:   library/std/src/sys/pal/unix/stack_overflow.rs
	modified:   library/std/src/sys/pal/unix/thread.rs
	modified:   library/std/src/sys/pal/windows/mod.rs
	modified:   library/std/src/sys/pal/xous/mod.rs
	modified:   library/std/src/sys/pal/xous/os.rs
	modified:   library/std/src/sys/process/unix/common.rs
	modified:   library/std/src/sys/process/unix/unix.rs
	modified:   library/std/src/sys/process/unix/vxworks.rs
	modified:   library/std/src/sys/process/windows.rs
	modified:   library/std/src/sys/sync/mutex/futex.rs
	modified:   library/std/src/sys/sync/mutex/pthread.rs
	modified:   library/std/src/sys/thread_local/guard/key.rs
	modified:   library/std/src/sys/thread_local/native/lazy.rs
	modified:   library/std/src/thread/mod.rs
	modified:   library/std/tests/floats/f128.rs
	modified:   library/std/tests/floats/f16.rs
	modified:   library/std/tests/floats/f32.rs
	modified:   library/std/tests/floats/f64.rs
	modified:   library/std/tests/floats/lib.rs
	modified:   library/std/tests/path.rs
	modified:   library/stdarch (modified content)

Untracked files:
  (use "git add <file>..." to include in what will be committed)
	bootstrap.toml.aleksey
	library/std/src/f128.rs
	library/std/src/f16.rs
	library/std/src/f32.rs
	library/std/src/f64.rs
	library/std/src/num.rs

no changes added to commit (use "git add" and/or "git commit -a")

Bootstrap configuration (bootstrap.toml)

# Use different pre-set defaults than the global defaults.
#
# See `src/bootstrap/defaults` for more information.
# Note that this has no default value (x.py uses the defaults in `bootstrap.example.toml`).
profile = 'dist'

[llvm]

# Currently, we only support this when building LLVM for the build triple.
#
# Note that many of the LLVM options are not currently supported for
# downloading. Currently only the "assertions" option can be toggled.
download-ci-llvm = true

[gcc]
# Download GCC from CI instead of building it locally.
# Note that this will attempt to download GCC even if there are local
# modifications to the `src/gcc` submodule.
# Currently, this is only supported for the `x86_64-unknown-linux-gnu` target.
#download-ci-gcc = false

[build]

# Instead of downloading the src/stage0 version of Cargo specified, use
# this Cargo binary instead to build all Rust code
# If you set this, you likely want to set `rustc` as well.
cargo = '/home/aleksey/.rustup/toolchains/1.88.0-beta.4-x86_64-unknown-linux-gnu/bin/cargo'

# Instead of downloading the src/stage0 version of the compiler
# specified, use this rustc binary instead as the stage0 snapshot compiler.
# If you set this, you likely want to set `cargo` as well.
rustc = '/home/aleksey/.rustup/toolchains/1.88.0-beta.4-x86_64-unknown-linux-gnu/bin/rustc'

# Arguments passed to the `./configure` script, used during distcheck. You
# probably won't fill this in but rather it's filled in by the `./configure`
# script. Useful for debugging.
configure-args = ['--set', 'build.cargo=/home/aleksey/.rustup/toolchains/1.88.0-beta.4-x86_64-unknown-linux-gnu/bin/cargo', '--set', 'build.rustc=/home/aleksey/.rustup/toolchains/1.88.0-beta.4-x86_64-unknown-linux-gnu/bin/rustc', '--set', 'rust.channel=nightly', '--set', 'profile=dist', '--set', 'llvm.download-ci-llvm=true', '--set', 'rust.download-rustc=false']

[install]

[rust]

# Whether to download the stage 1 and 2 compilers from CI. This is useful if you
# are working on tools, doc-comments, or library (you will be able to build the
# standard library without needing to build the compiler).
#
# Set this to "if-unchanged" if you are working on `src/tools`, `tests` or
# `library` (on CI, `library` changes triggers in-tree compiler build) to speed
# up the build process if you don't need to build a compiler from the latest
# commit from `master`.
#
# Set this to `true` to always download or `false` to always use the in-tree
# compiler.
download-rustc = false

# The "channel" for the Rust build to produce. The stable/beta channels only
# allow using stable features, whereas the nightly and dev channels allow using
# nightly features.
#
# You can set the channel to "auto-detect" to load the channel name from `src/ci/channel`.
#
# If using tarball sources, default value is "auto-detect", otherwise, it's "dev".
channel = 'nightly'

[target.x86_64-unknown-linux-gnu]

[dist]

Operating system

Observed on: Debian trixie (amd64), also macOS 15.5 (arm64)

HEAD

5d707b0

Additional context

At work we try to build nightlies using our in-house custom beta toolchain which includes a number of components (including 'rust-src') by default.

Build Log

log.txt

@lambdageek lambdageek added T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) C-bug Category: This is a bug. labels Jun 3, 2025
@rustbot rustbot added the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label Jun 3, 2025
@lambdageek
Copy link
Contributor Author

lambdageek commented Jun 4, 2025

Added some tracing and I see this:

          DEBUG bootstrap::core::build_steps::compile building compiler libraries to link to, build_compiler=Compiler { stage: 0, host: x86_64-unknown-linux-gnu, forced_compiler: false }, target_compiler.host=x86_64-unknown-linux-gnu
           bootstrap::core::build_steps::compile::Rustc::run previous_compiler=Compiler { stage: 0, host: x86_64-unknown-linux-gnu, forced_compiler: false }, target=x86_64-unknown-linux-gnu
             bootstrap::core::build_steps::compile::Std::run target=x86_64-unknown-linux-gnu, compiler=Compiler { stage: 0, host: x86_64-unknown-linux-gnu, forced_compiler: false }, force_recompile=false
               bootstrap::core::build_steps::compile::StdLink::run compiler=Compiler { stage: 0, host: x86_64-unknown-linux-gnu, forced_compiler: false }, target_compiler=Compiler { stage: 0, host: x86_64-unknown-linux-gnu, forced_compiler: false }, target=x86_64-unknown-linux-gnu
                 bootstrap::core::build_steps::compile::Sysroot::run compiler=Compiler { stage: 0, host: x86_64-unknown-linux-gnu, forced_compiler: false }
                  TRACE bootstrap::core::build_steps::compile stage=0, sysroot="/home/aleksey/hacking/rust-boot/rust/build/x86_64-unknown-linux-gnu/stage0-sysroot"
                   bootstrap::core::build_steps::dist::maybe_install_llvm_target llvm_link_shared=false, target=x86_64-unknown-linux-gnu, sysroot="/home/aleksey/hacking/rust-boot/rust/build/x86_64-unknown-linux-gnu/stage0-sysroot"
                  DEBUG bootstrap::core::build_steps::compile Creating Symlink  src: /home/aleksey/hacking/rust-boot/rust, sysroot_src: /home/aleksey/hacking/rust-boot/rust/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib/rustlib/src/rust
                DEBUG bootstrap::core::build_steps::compile StdLink::run stage0 case, cp_link_r src: /home/aleksey/.rustup/toolchains/1.88.0-beta.4-x86_64-unknown-linux-gnu/lib, dest: /home/aleksey/hacking/rust-boot/rust/build/x86_64-unknown-linux-gnu/stage0-sysroot/lib

"Creating Symlink" is a debug! right here:

if let Err(e) = symlink_dir(&builder.config, &builder.src, &sysroot_lib_rustlib_src_rust) {

and "StdLink::run stage0 case" is a debug! over here:

builder.cp_link_r(&builder.initial_sysroot.join("lib"), &sysroot.join("lib"));

So we first symlink the git repo root into stage0-sysroot/lib/rustlib/src/rust
and then we copy everything from .rustup/toolchains/1.88.0-beta.4-x86_64-unknown-linux-gnu/lib (including its rustlib/src/rust/ contents) into the sysroot and end up writing over the repo sources

My guess is we probably don't want to create that symlink if compiler.stage == 0 now, does that make sense?
Update yeah, this fixes it for me:

patch
diff --git a/src/bootstrap/src/core/build_steps/compile.rs b/src/bootstrap/src/core/build_steps/compile.rs
index 40247288e75..3729af3a2e5 100644
--- a/src/bootstrap/src/core/build_steps/compile.rs
+++ b/src/bootstrap/src/core/build_steps/compile.rs
@@ -1870,26 +1870,28 @@ fn run(self, builder: &Builder<'_>) -> PathBuf {
         // so that any tools relying on `rust-src` also work for local builds,
         // and also for translating the virtual `/rustc/$hash` back to the real
         // directory (for running tests with `rust.remap-debuginfo = true`).
-        let sysroot_lib_rustlib_src = sysroot.join("lib/rustlib/src");
-        t!(fs::create_dir_all(&sysroot_lib_rustlib_src));
-        let sysroot_lib_rustlib_src_rust = sysroot_lib_rustlib_src.join("rust");
-        if let Err(e) = symlink_dir(&builder.config, &builder.src, &sysroot_lib_rustlib_src_rust) {
-            eprintln!(
-                "ERROR: creating symbolic link `{}` to `{}` failed with {}",
-                sysroot_lib_rustlib_src_rust.display(),
-                builder.src.display(),
-                e,
-            );
-            if builder.config.rust_remap_debuginfo {
+        if compiler.stage != 0 {
+            let sysroot_lib_rustlib_src = sysroot.join("lib/rustlib/src");
+            t!(fs::create_dir_all(&sysroot_lib_rustlib_src));
+            let sysroot_lib_rustlib_src_rust = sysroot_lib_rustlib_src.join("rust");
+            if let Err(e) = symlink_dir(&builder.config, &builder.src, &sysroot_lib_rustlib_src_rust) {
                 eprintln!(
-                    "ERROR: some `tests/ui` tests will fail when lacking `{}`",
+                    "ERROR: creating symbolic link `{}` to `{}` failed with {}",
                     sysroot_lib_rustlib_src_rust.display(),
+                    builder.src.display(),
+                    e,
                 );
+                if builder.config.rust_remap_debuginfo {
+                    eprintln!(
+                        "ERROR: some `tests/ui` tests will fail when lacking `{}`",
+                        sysroot_lib_rustlib_src_rust.display(),
+                    );
+                }
+                build_helper::exit!(1);
             }
-            build_helper::exit!(1);
         }

@onur-ozkan onur-ozkan removed the needs-triage This issue may need triage. Remove it if it has been sufficiently triaged. label Jun 4, 2025
@onur-ozkan
Copy link
Member

That seems to make sense. Would you mind sending a PR?

lambdageek added a commit to lambdageek/upstream-rust that referenced this issue Jun 4, 2025
In StdLink::run we subsequently recursively copy the initial sysroot
lib directory into the stage0-sysroot lib directory.  If the initial
sysroot is a toolchain that includes the `rust-src` component (in
lib/rustlib/src/rust), if we add this symlink, that recursive copy we
will overwrite the repo sources with the toolchain's sources.

Fixes rust-lang#141991
@lambdageek
Copy link
Contributor Author

@onur-ozkan #142000

@bors bors closed this as completed in d31faac Jun 4, 2025
rust-timer added a commit that referenced this issue Jun 4, 2025
Rollup merge of #142000 - lambdageek:no-symlink-stage0, r=onur-ozkan

bootstrap: don't symlink source dir into stage0 sysroot

In StdLink::run we subsequently recursively copy the initial sysroot lib directory into the stage0-sysroot lib directory.  If the initial sysroot is a toolchain that includes the `rust-src` component (in lib/rustlib/src/rust), if we add this symlink, that recursive copy will overwrite the repo sources with the toolchain's sources.

Fixes #141991
github-actions bot pushed a commit to rust-lang/miri that referenced this issue Jun 5, 2025
bootstrap: don't symlink source dir into stage0 sysroot

In StdLink::run we subsequently recursively copy the initial sysroot lib directory into the stage0-sysroot lib directory.  If the initial sysroot is a toolchain that includes the `rust-src` component (in lib/rustlib/src/rust), if we add this symlink, that recursive copy will overwrite the repo sources with the toolchain's sources.

Fixes rust-lang/rust#141991
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-bug Category: This is a bug. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants