Skip to content

x.py fails with custom Git remote name #106651

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
ChaiTRex opened this issue Jan 9, 2023 · 4 comments
Closed

x.py fails with custom Git remote name #106651

ChaiTRex opened this issue Jan 9, 2023 · 4 comments
Labels
C-bug Category: This is a bug. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)

Comments

@ChaiTRex
Copy link
Contributor

ChaiTRex commented Jan 9, 2023

I tried running this:

$ git clone -o github [email protected]:rust-lang/rust
Cloning into 'rust'...
remote: Enumerating objects: 2121843, done.
remote: Counting objects: 100% (23/23), done.
remote: Total 2121843 (delta 22), reused 22 (delta 22), pack-reused 2121820
Receiving objects: 100% (2121843/2121843), 1022.07 MiB | 11.90 MiB/s, done.
Resolving deltas: 100% (1658141/1658141), done.

$ cd rust

$ git remote --verbose
github	[email protected]:rust-lang/rust (fetch)
github	[email protected]:rust-lang/rust (push)

$ ./configure
configure: processing command line
configure: 
configure: build.configure-args := []
configure: 
configure: writing `config.toml` in current directory
configure: 
configure: run `python /home/user/rust/x.py --help`

$ python x.py build
First failing `build` output 
downloading https://static.rust-lang.org/dist/2022-12-27/rust-std-beta-x86_64-unknown-linux-gnu.tar.xz
######################################################################################################################################################################################################################################################### 100.0%
extracting /home/user/rust/build/cache/2022-12-27/rust-std-beta-x86_64-unknown-linux-gnu.tar.xz
downloading https://static.rust-lang.org/dist/2022-12-27/rustc-beta-x86_64-unknown-linux-gnu.tar.xz
######################################################################################################################################################################################################################################################### 100.0%
extracting /home/user/rust/build/cache/2022-12-27/rustc-beta-x86_64-unknown-linux-gnu.tar.xz
downloading https://static.rust-lang.org/dist/2022-12-27/cargo-beta-x86_64-unknown-linux-gnu.tar.xz
######################################################################################################################################################################################################################################################### 100.0%
extracting /home/user/rust/build/cache/2022-12-27/cargo-beta-x86_64-unknown-linux-gnu.tar.xz
Building rustbuild
   Compiling libc v0.2.137
   Compiling memchr v2.5.0
   Compiling cfg-if v1.0.0
   Compiling proc-macro2 v1.0.46
   Compiling version_check v0.9.4
   Compiling typenum v1.15.0
   Compiling unicode-ident v1.0.0
   Compiling syn v1.0.102
   Compiling cc v1.0.73
   Compiling serde_derive v1.0.137
   Compiling io-lifetimes v1.0.1
   Compiling serde v1.0.137
   Compiling regex-automata v0.1.10
   Compiling lazy_static v1.4.0
   Compiling pkg-config v0.3.25
   Compiling log v0.4.17
   Compiling regex-syntax v0.6.26
   Compiling crossbeam-utils v0.8.14
   Compiling rustix v0.36.3
   Compiling once_cell v1.12.0
   Compiling fnv v1.0.7
   Compiling serde_json v1.0.81
   Compiling bitflags v1.3.2
   Compiling linux-raw-sys v0.1.3
   Compiling same-file v1.0.6
   Compiling ryu v1.0.10
   Compiling cpufeatures v0.2.5
   Compiling walkdir v2.3.2
   Compiling itoa v1.0.2
   Compiling unicode-width v0.1.9
   Compiling thread_local v1.1.4
   Compiling bootstrap v0.0.0 (/home/user/rust/src/bootstrap)
   Compiling getopts v0.2.21
   Compiling hex v0.4.3
   Compiling build_helper v0.1.0 (/home/user/rust/src/tools/build_helper)
   Compiling generic-array v0.14.5
   Compiling cmake v0.1.48
   Compiling lzma-sys v0.1.17
   Compiling quote v1.0.18
   Compiling bstr v0.2.17
   Compiling aho-corasick v0.7.18
   Compiling object v0.29.0
   Compiling opener v0.5.0
   Compiling xattr v0.2.3
   Compiling filetime v0.2.16
   Compiling tar v0.4.38
   Compiling crypto-common v0.1.3
   Compiling block-buffer v0.10.2
   Compiling digest v0.10.3
   Compiling sha2 v0.10.2
   Compiling regex v1.5.6
   Compiling globset v0.4.8
   Compiling fd-lock v3.0.8
   Compiling ignore v0.4.18
   Compiling xz2 v0.1.6
   Compiling toml v0.5.9
    Finished dev [unoptimized] target(s) in 9.90s
Updating submodule src/tools/rust-installer
Submodule 'src/rust-installer' (https://github.com/rust-lang/rust-installer.git) registered for path 'src/tools/rust-installer'
Cloning into '/home/user/rust/src/tools/rust-installer'...
remote: Enumerating objects: 53, done.        
remote: Counting objects: 100% (53/53), done.        
remote: Compressing objects: 100% (31/31), done.        
remote: Total 53 (delta 3), reused 38 (delta 0), pack-reused 0        
Receiving objects: 100% (53/53), 39.43 KiB | 3.29 MiB/s, done.
Resolving deltas: 100% (3/3), done.
Submodule path 'src/tools/rust-installer': checked out '5b2eee7eed72b4894909c5eecbf014ea0b5ad995'
Updating submodule src/tools/cargo
Submodule 'src/tools/cargo' (https://github.com/rust-lang/cargo.git) registered for path 'src/tools/cargo'
Cloning into '/home/user/rust/src/tools/cargo'...
remote: Enumerating objects: 1751, done.        
remote: Counting objects: 100% (1751/1751), done.        
remote: Compressing objects: 100% (1212/1212), done.        
remote: Total 1751 (delta 454), reused 1051 (delta 363), pack-reused 0        
Receiving objects: 100% (1751/1751), 2.44 MiB | 7.16 MiB/s, done.
Resolving deltas: 100% (454/454), done.
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
fatal: transport 'file' not allowed
fatal: Fetched in submodule path 'src/tools/cargo', but it did not contain 8c460b2237a6359a7e3335890db8da049bdd62fc. Direct fetching of that commit failed.
fatal: 
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
fatal: transport 'file' not allowed
fatal: Fetched in submodule path 'src/tools/cargo', but it did not contain 8c460b2237a6359a7e3335890db8da049bdd62fc. Direct fetching of that commit failed.
fatal: 
Build completed unsuccessfully in 0:00:31

$ python x.py build
Building rustbuild
    Finished dev [unoptimized] target(s) in 0.02s
Updating submodule src/tools/cargo
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
fatal: transport 'file' not allowed
fatal: Fetched in submodule path 'src/tools/cargo', but it did not contain 8c460b2237a6359a7e3335890db8da049bdd62fc. Direct fetching of that commit failed.
fatal: 
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
fatal: transport 'file' not allowed
fatal: Fetched in submodule path 'src/tools/cargo', but it did not contain 8c460b2237a6359a7e3335890db8da049bdd62fc. Direct fetching of that commit failed.
fatal: 
Build completed unsuccessfully in 0:00:01

$ python x.py test
Building rustbuild
    Finished dev [unoptimized] target(s) in 0.02s
Updating submodule src/tools/cargo
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
fatal: transport 'file' not allowed
fatal: Fetched in submodule path 'src/tools/cargo', but it did not contain 8c460b2237a6359a7e3335890db8da049bdd62fc. Direct fetching of that commit failed.
fatal: 
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
fatal: transport 'file' not allowed
fatal: Fetched in submodule path 'src/tools/cargo', but it did not contain 8c460b2237a6359a7e3335890db8da049bdd62fc. Direct fetching of that commit failed.
fatal: 
Build completed unsuccessfully in 0:00:00

$ python x.py doc
Building rustbuild
    Finished dev [unoptimized] target(s) in 0.02s
Updating submodule src/tools/cargo
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
fatal: transport 'file' not allowed
fatal: Fetched in submodule path 'src/tools/cargo', but it did not contain 8c460b2237a6359a7e3335890db8da049bdd62fc. Direct fetching of that commit failed.
fatal: 
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
fatal: transport 'file' not allowed
fatal: Fetched in submodule path 'src/tools/cargo', but it did not contain 8c460b2237a6359a7e3335890db8da049bdd62fc. Direct fetching of that commit failed.
fatal: 
Build completed unsuccessfully in 0:00:02

I expected to see this happen: Building, testing, and documenting should work fine when using a custom Git remote name

Instead, this happened: Building, testing, and documenting fail when using a custom Git remote name

Meta

rustc --version --verbose:

rustc 1.66.0 (69f9c33d7 2022-12-12)
binary: rustc
commit-hash: 69f9c33d71c871fc16ac445211281c6e7a340943
commit-date: 2022-12-12
host: x86_64-unknown-linux-gnu
release: 1.66.0
LLVM version: 15.0.2
@ChaiTRex ChaiTRex added the C-bug Category: This is a bug. label Jan 9, 2023
@ChaiTRex
Copy link
Contributor Author

ChaiTRex commented Jan 9, 2023

Workaround

Replace [custom-name] with the custom Git remote name and [github-user] with the GitHub user who owns the fork you're dealing with, and follow along with the commands below:

$ git remote --verbose
[custom-name]	[email protected]:[github-user]/rust (fetch)
[custom-name]	[email protected]:[github-user]/rust (push)

$ git remote remove [custom-name]

$ git remote add origin [email protected]:[github-user]/rust

$ git remote add [custom-name] [email protected]:[github-user]/rust

$ git remote --verbose
[custom-name]	[email protected]:[github-user]/rust (fetch)
[custom-name]	[email protected]:[github-user]/rust (push)
origin	[email protected]:[github-user]/rust (fetch)
origin	[email protected]:[github-user]/rust (push)

$ python x.py build

@Noratrieb
Copy link
Member

Can you check out some random older commit (like 1195b67) and see whether it's still broken there to see whether it's a recent regression?

@Noratrieb Noratrieb added the T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) label Jan 9, 2023
@ChaiTRex
Copy link
Contributor Author

ChaiTRex commented Jan 9, 2023 •
edited
Loading

No, I had issues with custom Git remote names years ago (I seem to remember trying to submit changes to configure, but I can't seem to find anything about it now).

With that particular commit:

$ git clone -o github [email protected]:rust-lang/rust
Cloning into 'rust'...
remote: Enumerating objects: 2121843, done.
remote: Counting objects: 100% (23/23), done.
remote: Total 2121843 (delta 22), reused 22 (delta 22), pack-reused 2121820
Receiving objects: 100% (2121843/2121843), 1022.07 MiB | 11.29 MiB/s, done.
Resolving deltas: 100% (1658141/1658141), done.

$ cd rust

$ git reset --hard  1195b67
HEAD is now at 1195b672fb5 Auto merge of #104757 - camelid:consolidate-lints, r=GuillaumeGomez,jyn514,Manishearth

$ ./configure
configure: processing command line
configure: 
configure: build.configure-args := []
configure: 
configure: writing `config.toml` in current directory
configure: 
configure: run `python /home/user/rust/x.py --help`
Builds that don't work due to missing build tools 
$ python x.py build
downloading https://static.rust-lang.org/dist/2022-11-01/rust-std-beta-x86_64-unknown-linux-gnu.tar.xz
#=#=-  #       #                                                                                                                                                                                                                                               #=O#-     #        #                                                                                                                                                                                                                                           -#O=- #      #          #         ######################################################################################################################################################################################################################################################### 100.0%
extracting /home/user/rust/build/cache/2022-11-01/rust-std-beta-x86_64-unknown-linux-gnu.tar.xz
downloading https://static.rust-lang.org/dist/2022-11-01/rustc-beta-x86_64-unknown-linux-gnu.tar.xz
#=#=-  #       #                                                                                                                                                                                                                                               #=O#-     #      ######################################################################################################################################################################################################################################################### 100.0%
extracting /home/user/rust/build/cache/2022-11-01/rustc-beta-x86_64-unknown-linux-gnu.tar.xz
downloading https://static.rust-lang.org/dist/2022-11-01/cargo-beta-x86_64-unknown-linux-gnu.tar.xz
######################################################################################################################################################################################################################################################### 100.0%
extracting /home/user/rust/build/cache/2022-11-01/cargo-beta-x86_64-unknown-linux-gnu.tar.xz
Building rustbuild
  Downloaded cpufeatures v0.2.2
  Downloaded crossbeam-utils v0.8.8
  Downloaded 2 crates (50.3 KB) in 0.26s
   Compiling libc v0.2.137
   Compiling memchr v2.5.0
   Compiling proc-macro2 v1.0.46
   Compiling cfg-if v1.0.0
   Compiling version_check v0.9.4
   Compiling unicode-ident v1.0.0
   Compiling typenum v1.15.0
   Compiling syn v1.0.102
   Compiling cc v1.0.73
   Compiling serde_derive v1.0.137
   Compiling lazy_static v1.4.0
   Compiling serde v1.0.137
   Compiling pkg-config v0.3.25
   Compiling io-lifetimes v1.0.1
   Compiling log v0.4.17
   Compiling regex-automata v0.1.10
   Compiling crossbeam-utils v0.8.8
   Compiling regex-syntax v0.6.26
   Compiling rustix v0.36.3
   Compiling fnv v1.0.7
   Compiling once_cell v1.12.0
   Compiling linux-raw-sys v0.1.3
   Compiling same-file v1.0.6
   Compiling bitflags v1.3.2
   Compiling serde_json v1.0.81
   Compiling ryu v1.0.10
   Compiling unicode-width v0.1.9
   Compiling cpufeatures v0.2.2
   Compiling itoa v1.0.2
   Compiling bootstrap v0.0.0 (/home/user/rust/src/bootstrap)
   Compiling walkdir v2.3.2
   Compiling getopts v0.2.21
   Compiling thread_local v1.1.4
   Compiling hex v0.4.3
   Compiling generic-array v0.14.5
   Compiling cmake v0.1.48
   Compiling lzma-sys v0.1.17
   Compiling bstr v0.2.17
   Compiling aho-corasick v0.7.18
   Compiling object v0.29.0
   Compiling quote v1.0.18
   Compiling opener v0.5.0
   Compiling xattr v0.2.3
   Compiling filetime v0.2.16
   Compiling tar v0.4.38
   Compiling crypto-common v0.1.3
   Compiling block-buffer v0.10.2
   Compiling digest v0.10.3
   Compiling regex v1.5.6
   Compiling sha2 v0.10.2
   Compiling globset v0.4.8
   Compiling ignore v0.4.18
   Compiling fd-lock v3.0.8
   Compiling xz2 v0.1.6
   Compiling toml v0.5.9
    Finished dev [unoptimized] target(s) in 10.28s

Couldn't find required command: cmake

You should install cmake, or set `download-ci-llvm = true` in the
`[llvm]` section section of `config.toml` to download LLVM rather
than building it.

Build completed unsuccessfully in 0:00:30

$ sudo apt install cmake
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following additional packages will be installed:
  cmake-data dh-elpa-helper libjsoncpp25 librhash0
Suggested packages:
  cmake-doc ninja-build cmake-format
The following NEW packages will be installed:
  cmake cmake-data dh-elpa-helper libjsoncpp25 librhash0
0 upgraded, 5 newly installed, 0 to remove and 44 not upgraded.
Need to get 7,138 kB of archives.
After this operation, 31.8 MB of additional disk space will be used.
Do you want to continue? [Y/n] 
Get:1 http://apt.pop-os.org/ubuntu jammy/main amd64 libjsoncpp25 amd64 1.9.5-3 [80.0 kB]
Get:2 http://apt.pop-os.org/ubuntu jammy/main amd64 librhash0 amd64 1.4.2-1ubuntu1 [125 kB]
Get:3 http://apt.pop-os.org/ubuntu jammy/main amd64 dh-elpa-helper all 2.0.9ubuntu1 [7,610 B]
Get:4 http://apt.pop-os.org/ubuntu jammy-updates/main amd64 cmake-data all 3.22.1-1ubuntu1.22.04.1 [1,913 kB]
Get:5 http://apt.pop-os.org/ubuntu jammy-updates/main amd64 cmake amd64 3.22.1-1ubuntu1.22.04.1 [5,013 kB]
Fetched 7,138 kB in 1s (9,489 kB/s)
Selecting previously unselected package libjsoncpp25:amd64.
(Reading database ... 282274 files and directories currently installed.)
Preparing to unpack .../libjsoncpp25_1.9.5-3_amd64.deb ...
Unpacking libjsoncpp25:amd64 (1.9.5-3) ...
Selecting previously unselected package librhash0:amd64.
Preparing to unpack .../librhash0_1.4.2-1ubuntu1_amd64.deb ...
Unpacking librhash0:amd64 (1.4.2-1ubuntu1) ...
Selecting previously unselected package dh-elpa-helper.
Preparing to unpack .../dh-elpa-helper_2.0.9ubuntu1_all.deb ...
Unpacking dh-elpa-helper (2.0.9ubuntu1) ...
Selecting previously unselected package cmake-data.
Preparing to unpack .../cmake-data_3.22.1-1ubuntu1.22.04.1_all.deb ...
Unpacking cmake-data (3.22.1-1ubuntu1.22.04.1) ...
Selecting previously unselected package cmake.
Preparing to unpack .../cmake_3.22.1-1ubuntu1.22.04.1_amd64.deb ...
Unpacking cmake (3.22.1-1ubuntu1.22.04.1) ...
Setting up dh-elpa-helper (2.0.9ubuntu1) ...
Setting up libjsoncpp25:amd64 (1.9.5-3) ...
Setting up librhash0:amd64 (1.4.2-1ubuntu1) ...
Setting up cmake-data (3.22.1-1ubuntu1.22.04.1) ...
Setting up cmake (3.22.1-1ubuntu1.22.04.1) ...
Processing triggers for man-db (2.10.2-1) ...
Processing triggers for libc-bin (2.35-0ubuntu3.1) ...

$ python x.py build
Building rustbuild
    Finished dev [unoptimized] target(s) in 0.02s

Couldn't find required command: ninja (or ninja-build)

You should install ninja as described at
<https://github.com/ninja-build/ninja/wiki/Pre-built-Ninja-packages>,
or set `ninja = false` in the `[llvm]` section of `config.toml`.
Alternatively, set `download-ci-llvm = true` in that `[llvm]` section
to download LLVM rather than building it.

Build completed unsuccessfully in 0:00:00

$ sudo apt install ninja-build
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
The following NEW packages will be installed:
  ninja-build
0 upgraded, 1 newly installed, 0 to remove and 55 not upgraded.
Need to get 111 kB of archives.
After this operation, 313 kB of additional disk space will be used.
Get:1 http://apt.pop-os.org/release jammy/main amd64 ninja-build amd64 1.10.1-1pop0~1630348646~22.04~fe200e8 [111 kB]
Fetched 111 kB in 0s (686 kB/s) 
Selecting previously unselected package ninja-build.
(Reading database ... 285360 files and directories currently installed.)
Preparing to unpack .../ninja-build_1.10.1-1pop0~1630348646~22.04~fe200e8_amd64.deb ...
Unpacking ninja-build (1.10.1-1pop0~1630348646~22.04~fe200e8) ...
Setting up ninja-build (1.10.1-1pop0~1630348646~22.04~fe200e8) ...
Processing triggers for man-db (2.10.2-1) ...

$ python x.py build
Building rustbuild
    Finished dev [unoptimized] target(s) in 0.02s
Updating submodule src/tools/rust-installer
Submodule 'src/rust-installer' (https://github.com/rust-lang/rust-installer.git) registered for path 'src/tools/rust-installer'
Cloning into '/home/user/rust/src/tools/rust-installer'...
remote: Enumerating objects: 53, done.        
remote: Counting objects: 100% (53/53), done.        
remote: Compressing objects: 100% (31/31), done.        
remote: Total 53 (delta 3), reused 38 (delta 0), pack-reused 0        
Receiving objects: 100% (53/53), 39.43 KiB | 1.07 MiB/s, done.
Resolving deltas: 100% (3/3), done.
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
fatal: transport 'file' not allowed
fatal: Fetched in submodule path 'src/tools/rust-installer', but it did not contain 300b5ec61ef38855a07e6bb4955a37aa1c414c00. Direct fetching of that commit failed.
fatal: 
remote: Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
fatal: transport 'file' not allowed
fatal: Fetched in submodule path 'src/tools/rust-installer', but it did not contain 300b5ec61ef38855a07e6bb4955a37aa1c414c00. Direct fetching of that commit failed.
fatal: 
Build completed unsuccessfully in 0:00:01

@jyn514
Copy link
Member

jyn514 commented Jan 10, 2023

Duplicate of #101144

@jyn514 jyn514 marked this as a duplicate of #101144 Jan 10, 2023
@jyn514 jyn514 closed this as not planned Won't fix, can't repro, duplicate, stale Jan 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
C-bug Category: This is a bug. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap)
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@ChaiTRex @jyn514 @Noratrieb