Fix miscompilation of inline assembly with outputs in cases where we emit an invoke instead of call instruction. #95864
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
We may sometimes emit an `invoke` instead of a `call` for inline assembly during the MIR -> LLVM IR lowering. But we failed to update the IR builder's current basic block before writing the results to the outputs. This would result in invalid IR because the basic block would end in a `store` instruction, which isn't a valid terminator.
rustbot
added
the
T-compiler
|
(rust-highfive has picked a reviewer for you, use r? to override) |
rust-highfive
added
the
S-waiting-on-review
|
r? @Amanieu |
|
Nice catch! And thanks for the detailed writeup, that is excellent work. I will merge this PR since it definitely fixes a bug in rustc, however while reviewing I noticed several other bugs that should also be fixed:
@bors r+ |
|
📌 Commit 0b2f360 has been approved by |
bors
added
S-waiting-on-bors
|
No problem! One minor thing I realize I forgot to update was a misleading comment on the cleanup field of rust/compiler/rustc_middle/src/mir/terminator.rs Lines 271 to 273 in e9a0c42 Outside of the initial MIR construction, subsequent passes may end up rendering that statement false. |
This was referenced Apr 10, 2022
Dylan-DPC
added a commit
to Dylan-DPC/rust
that referenced
this pull request
Apr 10, 2022
…compile, r=Amanieu
Fix miscompilation of inline assembly with outputs in cases where we emit an invoke instead of call instruction.
We ran into this bug where rustc would segfault while trying to compile certain uses of inline assembly.
Here is a simple repro that demonstrates the issue:
```rust
#![feature(asm_unwind)]
fn main() {
let _x = String::from("string here just cause we need something with a non-trivial drop");
let foo: u64;
unsafe {
std::arch::asm!(
"mov {}, 1",
out(reg) foo,
options(may_unwind)
);
}
println!("{}", foo);
}
```
([playground link](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=7d6641e83370d2536a07234aca2498ff))
But crucially `feature(asm_unwind)` is not actually needed and this can be triggered on stable as a result of the way async functions/generators are handled in the compiler. e.g.:
```rust
extern crate futures; // 0.3.21
async fn bar() {
let foo: u64;
unsafe {
std::arch::asm!(
"mov {}, 1",
out(reg) foo,
);
}
println!("{}", foo);
}
fn main() {
futures::executor::block_on(bar());
}
```
([playground link](https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=1c7781c34dd4a3e80ae4bd936a0c82fc))
An example of the incorrect LLVM generated:
```llvm
bb1: ; preds = %start
%1 = invoke i64 asm sideeffect alignstack inteldialect unwind "mov ${0:q}, 1", "=&r,~{dirflag},~{fpsr},~{flags},~{memory}"()
to label %bb2 unwind label %cleanup, !srcloc !9
store i64 %1, i64* %foo, align 8
bb2:
[...snip...]
```
The store should not be placed after the asm invoke but rather should be in the normal control flow basic block (`bb2` in this case).
[Here](https://gist.github.com/luqmana/be1af5b64d2cda5a533e3e23a7830b44) is a writeup of the investigation that lead to finding this.
Dylan-DPC
added a commit
to Dylan-DPC/rust
that referenced
this pull request
Apr 11, 2022
…compile, r=Amanieu
Fix miscompilation of inline assembly with outputs in cases where we emit an invoke instead of call instruction.
We ran into this bug where rustc would segfault while trying to compile certain uses of inline assembly.
Here is a simple repro that demonstrates the issue:
```rust
#![feature(asm_unwind)]
fn main() {
let _x = String::from("string here just cause we need something with a non-trivial drop");
let foo: u64;
unsafe {
std::arch::asm!(
"mov {}, 1",
out(reg) foo,
options(may_unwind)
);
}
println!("{}", foo);
}
```
([playground link](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=7d6641e83370d2536a07234aca2498ff))
But crucially `feature(asm_unwind)` is not actually needed and this can be triggered on stable as a result of the way async functions/generators are handled in the compiler. e.g.:
```rust
extern crate futures; // 0.3.21
async fn bar() {
let foo: u64;
unsafe {
std::arch::asm!(
"mov {}, 1",
out(reg) foo,
);
}
println!("{}", foo);
}
fn main() {
futures::executor::block_on(bar());
}
```
([playground link](https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=1c7781c34dd4a3e80ae4bd936a0c82fc))
An example of the incorrect LLVM generated:
```llvm
bb1: ; preds = %start
%1 = invoke i64 asm sideeffect alignstack inteldialect unwind "mov ${0:q}, 1", "=&r,~{dirflag},~{fpsr},~{flags},~{memory}"()
to label %bb2 unwind label %cleanup, !srcloc !9
store i64 %1, i64* %foo, align 8
bb2:
[...snip...]
```
The store should not be placed after the asm invoke but rather should be in the normal control flow basic block (`bb2` in this case).
[Here](https://gist.github.com/luqmana/be1af5b64d2cda5a533e3e23a7830b44) is a writeup of the investigation that lead to finding this.
Dylan-DPC
added a commit
to Dylan-DPC/rust
that referenced
this pull request
Apr 11, 2022
…compile, r=Amanieu
Fix miscompilation of inline assembly with outputs in cases where we emit an invoke instead of call instruction.
We ran into this bug where rustc would segfault while trying to compile certain uses of inline assembly.
Here is a simple repro that demonstrates the issue:
```rust
#![feature(asm_unwind)]
fn main() {
let _x = String::from("string here just cause we need something with a non-trivial drop");
let foo: u64;
unsafe {
std::arch::asm!(
"mov {}, 1",
out(reg) foo,
options(may_unwind)
);
}
println!("{}", foo);
}
```
([playground link](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=7d6641e83370d2536a07234aca2498ff))
But crucially `feature(asm_unwind)` is not actually needed and this can be triggered on stable as a result of the way async functions/generators are handled in the compiler. e.g.:
```rust
extern crate futures; // 0.3.21
async fn bar() {
let foo: u64;
unsafe {
std::arch::asm!(
"mov {}, 1",
out(reg) foo,
);
}
println!("{}", foo);
}
fn main() {
futures::executor::block_on(bar());
}
```
([playground link](https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=1c7781c34dd4a3e80ae4bd936a0c82fc))
An example of the incorrect LLVM generated:
```llvm
bb1: ; preds = %start
%1 = invoke i64 asm sideeffect alignstack inteldialect unwind "mov ${0:q}, 1", "=&r,~{dirflag},~{fpsr},~{flags},~{memory}"()
to label %bb2 unwind label %cleanup, !srcloc !9
store i64 %1, i64* %foo, align 8
bb2:
[...snip...]
```
The store should not be placed after the asm invoke but rather should be in the normal control flow basic block (`bb2` in this case).
[Here](https://gist.github.com/luqmana/be1af5b64d2cda5a533e3e23a7830b44) is a writeup of the investigation that lead to finding this.
Dylan-DPC
added a commit
to Dylan-DPC/rust
that referenced
this pull request
Apr 11, 2022
…compile, r=Amanieu
Fix miscompilation of inline assembly with outputs in cases where we emit an invoke instead of call instruction.
We ran into this bug where rustc would segfault while trying to compile certain uses of inline assembly.
Here is a simple repro that demonstrates the issue:
```rust
#![feature(asm_unwind)]
fn main() {
let _x = String::from("string here just cause we need something with a non-trivial drop");
let foo: u64;
unsafe {
std::arch::asm!(
"mov {}, 1",
out(reg) foo,
options(may_unwind)
);
}
println!("{}", foo);
}
```
([playground link](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=7d6641e83370d2536a07234aca2498ff))
But crucially `feature(asm_unwind)` is not actually needed and this can be triggered on stable as a result of the way async functions/generators are handled in the compiler. e.g.:
```rust
extern crate futures; // 0.3.21
async fn bar() {
let foo: u64;
unsafe {
std::arch::asm!(
"mov {}, 1",
out(reg) foo,
);
}
println!("{}", foo);
}
fn main() {
futures::executor::block_on(bar());
}
```
([playground link](https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=1c7781c34dd4a3e80ae4bd936a0c82fc))
An example of the incorrect LLVM generated:
```llvm
bb1: ; preds = %start
%1 = invoke i64 asm sideeffect alignstack inteldialect unwind "mov ${0:q}, 1", "=&r,~{dirflag},~{fpsr},~{flags},~{memory}"()
to label %bb2 unwind label %cleanup, !srcloc !9
store i64 %1, i64* %foo, align 8
bb2:
[...snip...]
```
The store should not be placed after the asm invoke but rather should be in the normal control flow basic block (`bb2` in this case).
[Here](https://gist.github.com/luqmana/be1af5b64d2cda5a533e3e23a7830b44) is a writeup of the investigation that lead to finding this.
Dylan-DPC
added a commit
to Dylan-DPC/rust
that referenced
this pull request
Apr 11, 2022
…compile, r=Amanieu
Fix miscompilation of inline assembly with outputs in cases where we emit an invoke instead of call instruction.
We ran into this bug where rustc would segfault while trying to compile certain uses of inline assembly.
Here is a simple repro that demonstrates the issue:
```rust
#![feature(asm_unwind)]
fn main() {
let _x = String::from("string here just cause we need something with a non-trivial drop");
let foo: u64;
unsafe {
std::arch::asm!(
"mov {}, 1",
out(reg) foo,
options(may_unwind)
);
}
println!("{}", foo);
}
```
([playground link](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=7d6641e83370d2536a07234aca2498ff))
But crucially `feature(asm_unwind)` is not actually needed and this can be triggered on stable as a result of the way async functions/generators are handled in the compiler. e.g.:
```rust
extern crate futures; // 0.3.21
async fn bar() {
let foo: u64;
unsafe {
std::arch::asm!(
"mov {}, 1",
out(reg) foo,
);
}
println!("{}", foo);
}
fn main() {
futures::executor::block_on(bar());
}
```
([playground link](https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=1c7781c34dd4a3e80ae4bd936a0c82fc))
An example of the incorrect LLVM generated:
```llvm
bb1: ; preds = %start
%1 = invoke i64 asm sideeffect alignstack inteldialect unwind "mov ${0:q}, 1", "=&r,~{dirflag},~{fpsr},~{flags},~{memory}"()
to label %bb2 unwind label %cleanup, !srcloc !9
store i64 %1, i64* %foo, align 8
bb2:
[...snip...]
```
The store should not be placed after the asm invoke but rather should be in the normal control flow basic block (`bb2` in this case).
[Here](https://gist.github.com/luqmana/be1af5b64d2cda5a533e3e23a7830b44) is a writeup of the investigation that lead to finding this.
Dylan-DPC
added a commit
to Dylan-DPC/rust
that referenced
this pull request
Apr 11, 2022
…compile, r=Amanieu
Fix miscompilation of inline assembly with outputs in cases where we emit an invoke instead of call instruction.
We ran into this bug where rustc would segfault while trying to compile certain uses of inline assembly.
Here is a simple repro that demonstrates the issue:
```rust
#![feature(asm_unwind)]
fn main() {
let _x = String::from("string here just cause we need something with a non-trivial drop");
let foo: u64;
unsafe {
std::arch::asm!(
"mov {}, 1",
out(reg) foo,
options(may_unwind)
);
}
println!("{}", foo);
}
```
([playground link](https://play.rust-lang.org/?version=nightly&mode=debug&edition=2021&gist=7d6641e83370d2536a07234aca2498ff))
But crucially `feature(asm_unwind)` is not actually needed and this can be triggered on stable as a result of the way async functions/generators are handled in the compiler. e.g.:
```rust
extern crate futures; // 0.3.21
async fn bar() {
let foo: u64;
unsafe {
std::arch::asm!(
"mov {}, 1",
out(reg) foo,
);
}
println!("{}", foo);
}
fn main() {
futures::executor::block_on(bar());
}
```
([playground link](https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=1c7781c34dd4a3e80ae4bd936a0c82fc))
An example of the incorrect LLVM generated:
```llvm
bb1: ; preds = %start
%1 = invoke i64 asm sideeffect alignstack inteldialect unwind "mov ${0:q}, 1", "=&r,~{dirflag},~{fpsr},~{flags},~{memory}"()
to label %bb2 unwind label %cleanup, !srcloc !9
store i64 %1, i64* %foo, align 8
bb2:
[...snip...]
```
The store should not be placed after the asm invoke but rather should be in the normal control flow basic block (`bb2` in this case).
[Here](https://gist.github.com/luqmana/be1af5b64d2cda5a533e3e23a7830b44) is a writeup of the investigation that lead to finding this.
This was referenced Apr 11, 2022
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Apr 11, 2022
Rollup of 7 pull requests Successful merges: - rust-lang#95008 ([`let_chains`] Forbid `let` inside parentheses) - rust-lang#95801 (Replace RwLock by a futex based one on Linux) - rust-lang#95864 (Fix miscompilation of inline assembly with outputs in cases where we emit an invoke instead of call instruction.) - rust-lang#95894 (Fix formatting error in pin.rs docs) - rust-lang#95895 (Clarify str::from_utf8_unchecked's invariants) - rust-lang#95901 (Remove duplicate aliases for `check codegen_{cranelift,gcc}` and fix `build codegen_gcc`) - rust-lang#95927 (CI: do not compile libcore twice when performing LLVM PGO) Failed merges: r? `@ghost` `@rustbot` modify labels: rollup
Dylan-DPC
added a commit
to Dylan-DPC/rust
that referenced
this pull request
Apr 12, 2022
…s, r=nikic Respect -Z verify-llvm-ir and other flags that add extra passes when combined with -C no-prepopulate-passes in the new LLVM Pass Manager. As part of the switch to the new LLVM Pass Manager the behaviour of flags such as `-Z verify-llvm-ir` (e.g. sanitizer, instrumentation) was modified when combined with `-C no-prepopulate-passes`. With the old PM, rustc was the one manually constructing the pipeline and respected those flags but in the new pass manager, those flags are used to build a list of callbacks that get invoked at certain extension points in the pipeline. Unfortunately, `-C no-prepopulate-passes` would skip building the pipeline altogether meaning we'd never add the corresponding passes. The fix here is to just manually invoke those callbacks as needed. Fixes rust-lang#95874 Demonstrating the current vs fixed behaviour using the bug in rust-lang#95864 ```console $ rustc +nightly asm-miscompile.rs --edition 2021 --emit=llvm-ir -C no-prepopulate-passes -Z verify-llvm-ir $ echo $? 0 $ rustc +stage1 asm-miscompile.rs --edition 2021 --emit=llvm-ir -C no-prepopulate-passes -Z verify-llvm-ir Basic Block in function '_ZN14asm_miscompile3foo28_$u7b$$u7b$closure$u7d$$u7d$17h360e2f7eee1275c5E' does not have terminator! label %bb1 LLVM ERROR: Broken module found, compilation aborted! ```
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
Apr 12, 2022
… r=nikic Respect -Z verify-llvm-ir and other flags that add extra passes when combined with -C no-prepopulate-passes in the new LLVM Pass Manager. As part of the switch to the new LLVM Pass Manager the behaviour of flags such as `-Z verify-llvm-ir` (e.g. sanitizer, instrumentation) was modified when combined with `-C no-prepopulate-passes`. With the old PM, rustc was the one manually constructing the pipeline and respected those flags but in the new pass manager, those flags are used to build a list of callbacks that get invoked at certain extension points in the pipeline. Unfortunately, `-C no-prepopulate-passes` would skip building the pipeline altogether meaning we'd never add the corresponding passes. The fix here is to just manually invoke those callbacks as needed. Fixes rust-lang#95874 Demonstrating the current vs fixed behaviour using the bug in rust-lang#95864 ```console $ rustc +nightly asm-miscompile.rs --edition 2021 --emit=llvm-ir -C no-prepopulate-passes -Z verify-llvm-ir $ echo $? 0 $ rustc +stage1 asm-miscompile.rs --edition 2021 --emit=llvm-ir -C no-prepopulate-passes -Z verify-llvm-ir Basic Block in function '_ZN14asm_miscompile3foo28_$u7b$$u7b$closure$u7d$$u7d$17h360e2f7eee1275c5E' does not have terminator! label %bb1 LLVM ERROR: Broken module found, compilation aborted! ```
JohnTitor
added a commit
to JohnTitor/rust
that referenced
this pull request
May 18, 2022
…eu,tmiasko Update MIR passes to handle unwinding Inline Asm Some more follow up fixes from rust-lang#95864 (comment) r? `@Amanieu`
bors
added a commit
to rust-lang-ci/rust
that referenced
this pull request
May 19, 2022
…,tmiasko Update MIR passes to handle unwinding Inline Asm Some more follow up fixes from rust-lang#95864 (comment) r? `@Amanieu`
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
We ran into this bug where rustc would segfault while trying to compile certain uses of inline assembly.
Here is a simple repro that demonstrates the issue:
(playground link)
But crucially
feature(asm_unwind)is not actually needed and this can be triggered on stable as a result of the way async functions/generators are handled in the compiler. e.g.:(playground link)
An example of the incorrect LLVM generated:
The store should not be placed after the asm invoke but rather should be in the normal control flow basic block (
bb2in this case).Here is a writeup of the investigation that lead to finding this.