Skip to content

Simplify CI LLVM checks in bootstrap #138704

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Mar 21, 2025
Merged

Conversation

Kobzol
Copy link
Contributor

@Kobzol Kobzol commented Mar 19, 2025

Extracted out of #138591. Apart from simplifying the checks, it will make it easier to run E2E tests of bootstrap on a mostly empty directory without checking out LLVM on CI :)

The commits should be mostly self-explanatory.

r? @onur-ozkan

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels Mar 19, 2025
@rustbot
Copy link
Collaborator

rustbot commented Mar 19, 2025

This PR changes how GCC is built. Consider updating src/bootstrap/download-ci-gcc-stamp.

This PR changes how LLVM is built. Consider updating src/bootstrap/download-ci-llvm-stamp.

This PR modifies src/bootstrap/src/core/config.

If appropriate, please update CONFIG_CHANGE_HISTORY in src/bootstrap/src/utils/change_tracker.rs.

@rust-log-analyzer

This comment has been minimized.

@Kobzol
Copy link
Contributor Author

Kobzol commented Mar 19, 2025

Uhh, crap, this breaks the check-default-config-profiles.sh smoke test, because it tries to check each profile on CI, and the library profile uses download-ci-llvm...

@rust-log-analyzer

This comment has been minimized.

@Kobzol
Copy link
Contributor Author

Kobzol commented Mar 19, 2025

Hmm, a lot of bootstrap tests parse flags to create a config, and the default value for download-ci-llvm is true. I'm not necessarily sure if that's a good default - shouldn't it be if-unchanged?

@onur-ozkan
Copy link
Member

"if-unchanged" syncs llvm submodule, that's why true is the default value.

@Kobzol Kobzol force-pushed the ci-llvm-checks branch 2 times, most recently from c349e6b to 59c3417 Compare March 20, 2025 12:58
@rustbot rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue. labels Mar 20, 2025
@Kobzol
Copy link
Contributor Author

Kobzol commented Mar 20, 2025

Rebased on top of #138743.

@rustbot blocked #138743

@rustbot rustbot added S-blocked Status: Blocked on something else such as an RFC or other implementation work. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Mar 20, 2025
@rust-log-analyzer

This comment has been minimized.

Kobzol added 5 commits March 21, 2025 12:18
Before it was using a different set of paths in different call-sites.
To avoid working around some code being unused in tests due to it being stubbed out with `#[cfg(test)]`.
@Kobzol Kobzol removed the S-blocked Status: Blocked on something else such as an RFC or other implementation work. label Mar 21, 2025
@onur-ozkan
Copy link
Member

Changes look good but this is one of those PRs where it's hard to find issues without merging it... So let's try that now.

@bors r+ rollup=never

@bors
Copy link
Collaborator

bors commented Mar 21, 2025

📌 Commit f5c59a4 has been approved by onur-ozkan

It is now in the queue for this repository.

@bors bors added the S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. label Mar 21, 2025
@bors
Copy link
Collaborator

bors commented Mar 21, 2025

⌛ Testing commit f5c59a4 with merge a4a11ac...

@bors
Copy link
Collaborator

bors commented Mar 21, 2025

☀️ Test successful - checks-actions
Approved by: onur-ozkan
Pushing a4a11ac to master...

@bors bors added the merged-by-bors This PR was explicitly merged by bors. label Mar 21, 2025
@bors bors merged commit a4a11ac into rust-lang:master Mar 21, 2025
7 checks passed
@rustbot rustbot added this to the 1.87.0 milestone Mar 21, 2025
Copy link

What is this? This is an experimental post-merge analysis report that shows differences in test outcomes between the merged PR and its parent PR.

Comparing 4ac032f (parent) -> a4a11ac (this PR)

Test differences

No test diffs found

@Kobzol Kobzol deleted the ci-llvm-checks branch March 21, 2025 18:17
@rust-timer
Copy link
Collaborator

Finished benchmarking commit (a4a11ac): comparison URL.

Overall result: ❌✅ regressions and improvements - no action needed

@rustbot label: -perf-regression

Instruction count

This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
0.3% [0.3%, 0.3%] 1
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-1.1% [-1.1%, -1.1%] 1
All ❌✅ (primary) - - 0

Max RSS (memory usage)

Results (secondary 3.7%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
3.7% [3.5%, 4.0%] 2
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) - - 0

Cycles

Results (primary -0.8%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
-0.8% [-0.8%, -0.8%] 1
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) -0.8% [-0.8%, -0.8%] 1

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 775.474s -> 775.351s (-0.02%)
Artifact size: 365.53 MiB -> 365.55 MiB (0.00%)

jhpratt added a commit to jhpratt/rust that referenced this pull request Mar 27, 2025
…ozkan

Remove unneeded LLVM CI test assertions

The `download_ci_llvm` bootstrap test was checking implementation details of the LLVM CI download check, which isn't very useful. It was essentially testing "if function_that_checks_if_llvm_ci_is_available returns true, we enable CI LLVM", but the usage of the function was an implementation detail. After rust-lang#138704, the inner implementation has changed, so the test now breaks if LLVM is updated.

I don't think that it's very useful to test implementation details like this, without taking the outside git state into account. Ideally, we should mock the git state for the test, otherwise the test will randomly break when executed in environments which the test does not control (e.g. on CI when a LLVM change happens).

I only kept the part of the test that checks that LLVM CI isn't used when we specify `download-ci-llvm = false`, as that should hold under all conditions, CI/local, and all git states.

I also kept the `if-unchanged` assertion, but only on CI, and as a temporary measure. After rust-lang#138591, we should have a proper way of mocking the git state to make the test robust, and make it test what we actually want.

Fixes [this](rust-lang#138784 (comment)).

r? `@ghost`
rust-timer added a commit to rust-lang-ci/rust that referenced this pull request Mar 27, 2025
Rollup merge of rust-lang#139015 - Kobzol:llvm-ci-test-fixes, r=onur-ozkan

Remove unneeded LLVM CI test assertions

The `download_ci_llvm` bootstrap test was checking implementation details of the LLVM CI download check, which isn't very useful. It was essentially testing "if function_that_checks_if_llvm_ci_is_available returns true, we enable CI LLVM", but the usage of the function was an implementation detail. After rust-lang#138704, the inner implementation has changed, so the test now breaks if LLVM is updated.

I don't think that it's very useful to test implementation details like this, without taking the outside git state into account. Ideally, we should mock the git state for the test, otherwise the test will randomly break when executed in environments which the test does not control (e.g. on CI when a LLVM change happens).

I only kept the part of the test that checks that LLVM CI isn't used when we specify `download-ci-llvm = false`, as that should hold under all conditions, CI/local, and all git states.

I also kept the `if-unchanged` assertion, but only on CI, and as a temporary measure. After rust-lang#138591, we should have a proper way of mocking the git state to make the test robust, and make it test what we actually want.

Fixes [this](rust-lang#138784 (comment)).

r? `@ghost`
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-testsuite Area: The testsuite used to check the correctness of rustc merged-by-bors This PR was explicitly merged by bors. S-waiting-on-bors Status: Waiting on bors to run and complete tests. Bors will change the label on completion. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) T-infra Relevant to the infrastructure team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants