Clang test Analysis/live-stmts.cpp randomly fails on MacOS #126804
Assignees
Labels
Comments
EugeneZelenko
added
test-suite
clang:static analyzer
and removed
clang
|
@llvm/issue-subscribers-clang-static-analyzer Author: None (dyung)
I have 2 build bots setup to build/test LLVM running on MacOS and I have noticed that the test `clang/test/Analysis/live-stmts.cpp` randomly seems to fail on the bot. I have two identically configured workers running the job, and it has failed on both so it doesn't seem specific to one machine configuration.
Here is a sample of the failing test output: Here are some recent runs of the build bot where the test failure occurred: |
|
Duplicate of #126619 |
|
Ill have a look at his |
|
I tried running this test about half a million times, and it always passed. I'm on linux x86_64. After looking at the output you observed, my guess is that there is some nondeterminism somewhere, making the dumps unstable wrt. their ordering. If you look at each I think this shouldn't be too hard to fix while dumping these, but we should maybe go to the bottom of this and consider making the backing implementation deterministic instead/in addition to sorting in the dumps. I don't believe it's the #100745 fault. I think it just exposed this behavior more often due to the additional tests. |
steakhal
added a commit
to steakhal/llvm-project
that referenced
this issue
Feb 12, 2025
Multiple people reported flaky bot failures tied to clang/test/Analysis/live-stmts.cpp I tried reproducing the flaky behavior on my Linux x86_64 system, but the tests appears to be stable in my context. Only by looking at the failures reported, I could formulate a potential diagnosis. The output always looked almost the same, except that the Exprs dumped per Basic block were shuffled compared to my expectation. This suggests to me some ordering issue. If you look at the backing storage of `blocksEndToLiveness[B].liveExprs`, it uses `llvm::ImmutableSet<const Expr *>`. That container likely uses the pointer values as keys, thus the runtime values of the addresses influence the iteration order. To fix this, before dumping, I sort the expressions by their "beginLocs". It should be efficient enough for a debug checker, where there is no performance constraint. This should hopefully fix the flaky behavior on systems where ASLR works differently than (my) Linux system. Hopefully fixes llvm#126619 Hopefully fixes llvm#126804
|
Find the fix at #126913 |
flovent
pushed a commit
to flovent/llvm-project
that referenced
this issue
Feb 13, 2025
…lvm#126913) Multiple people reported flaky bot failures tied to `clang/test/Analysis/live-stmts.cpp` I tried reproducing the flaky behavior on my Linux x86_64 system, but the tests appears to be stable in my context. Only by looking at the failures reported, I could formulate a potential diagnosis. The output always looked almost the same, except that the Exprs dumped per Basic block were shuffled compared to my expectation. This suggests to me some ordering issue. If you look at the backing storage of `blocksEndToLiveness[B].liveExprs`, it uses `llvm::ImmutableSet<const Expr *>`. That container likely uses the pointer values as keys, thus the runtime values of the addresses influence the iteration order. To fix this, before dumping, I sort the expressions by their "beginLocs". It should be efficient enough for a debug checker, where there is no performance constraint. This should hopefully fix the flaky behavior on systems where ASLR works differently than (my) Linux system. Hopefully fixes llvm#126619 Hopefully fixes llvm#126804
joaosaffran
pushed a commit
to joaosaffran/llvm-project
that referenced
this issue
Feb 14, 2025
…lvm#126913) Multiple people reported flaky bot failures tied to `clang/test/Analysis/live-stmts.cpp` I tried reproducing the flaky behavior on my Linux x86_64 system, but the tests appears to be stable in my context. Only by looking at the failures reported, I could formulate a potential diagnosis. The output always looked almost the same, except that the Exprs dumped per Basic block were shuffled compared to my expectation. This suggests to me some ordering issue. If you look at the backing storage of `blocksEndToLiveness[B].liveExprs`, it uses `llvm::ImmutableSet<const Expr *>`. That container likely uses the pointer values as keys, thus the runtime values of the addresses influence the iteration order. To fix this, before dumping, I sort the expressions by their "beginLocs". It should be efficient enough for a debug checker, where there is no performance constraint. This should hopefully fix the flaky behavior on systems where ASLR works differently than (my) Linux system. Hopefully fixes llvm#126619 Hopefully fixes llvm#126804
steakhal
added a commit
to steakhal/llvm-project
that referenced
this issue
Feb 16, 2025
…2nd attempt) In my previous attempt (llvm#126913) of fixing the flaky case was on a good track when I used the begin locations as a stable ordering. However, I forgot to consider the case when the begin locations are the same among the Exprs. In an `EXPENSIVE_CHECKS` build, arrays are randomly shuffled prior to sorting them. This exposed the flaky behavior much more often basically breaking the "stability" of the vector - as it should. To fix this, I I use this time `Expr::getID` for a stable ID for an Expr. Hopefully fixes llvm#126619 Hopefully fixes llvm#126804
steakhal
added a commit
that referenced
this issue
Feb 17, 2025
…2nd attempt) (#127406) In my previous attempt (#126913) of fixing the flaky case was on a good track when I used the begin locations as a stable ordering. However, I forgot to consider the case when the begin locations are the same among the Exprs. In an `EXPENSIVE_CHECKS` build, arrays are randomly shuffled prior to sorting them. This exposed the flaky behavior much more often basically breaking the "stability" of the vector - as it should. Because of this, I had to revert the previous fix attempt in #127034. To fix this, I use this time `Expr::getID` for a stable ID for an Expr. Hopefully fixes #126619 Hopefully fixes #126804
sivan-shani
pushed a commit
to sivan-shani/llvm-project
that referenced
this issue
Feb 24, 2025
…lvm#126913) Multiple people reported flaky bot failures tied to `clang/test/Analysis/live-stmts.cpp` I tried reproducing the flaky behavior on my Linux x86_64 system, but the tests appears to be stable in my context. Only by looking at the failures reported, I could formulate a potential diagnosis. The output always looked almost the same, except that the Exprs dumped per Basic block were shuffled compared to my expectation. This suggests to me some ordering issue. If you look at the backing storage of `blocksEndToLiveness[B].liveExprs`, it uses `llvm::ImmutableSet<const Expr *>`. That container likely uses the pointer values as keys, thus the runtime values of the addresses influence the iteration order. To fix this, before dumping, I sort the expressions by their "beginLocs". It should be efficient enough for a debug checker, where there is no performance constraint. This should hopefully fix the flaky behavior on systems where ASLR works differently than (my) Linux system. Hopefully fixes llvm#126619 Hopefully fixes llvm#126804
sivan-shani
pushed a commit
to sivan-shani/llvm-project
that referenced
this issue
Feb 24, 2025
…2nd attempt) (llvm#127406) In my previous attempt (llvm#126913) of fixing the flaky case was on a good track when I used the begin locations as a stable ordering. However, I forgot to consider the case when the begin locations are the same among the Exprs. In an `EXPENSIVE_CHECKS` build, arrays are randomly shuffled prior to sorting them. This exposed the flaky behavior much more often basically breaking the "stability" of the vector - as it should. Because of this, I had to revert the previous fix attempt in llvm#127034. To fix this, I use this time `Expr::getID` for a stable ID for an Expr. Hopefully fixes llvm#126619 Hopefully fixes llvm#126804
steakhal
added a commit
to steakhal/llvm-project
that referenced
this issue
May 12, 2025
…2nd attempt) (llvm#127406) In my previous attempt (llvm#126913) of fixing the flaky case was on a good track when I used the begin locations as a stable ordering. However, I forgot to consider the case when the begin locations are the same among the Exprs. In an `EXPENSIVE_CHECKS` build, arrays are randomly shuffled prior to sorting them. This exposed the flaky behavior much more often basically breaking the "stability" of the vector - as it should. Because of this, I had to revert the previous fix attempt in llvm#127034. To fix this, I use this time `Expr::getID` for a stable ID for an Expr. Hopefully fixes llvm#126619 Hopefully fixes llvm#126804 (cherry picked from commit f378e52)
swift-ci
pushed a commit
to swiftlang/llvm-project
that referenced
this issue
May 13, 2025
…2nd attempt) (llvm#127406) In my previous attempt (llvm#126913) of fixing the flaky case was on a good track when I used the begin locations as a stable ordering. However, I forgot to consider the case when the begin locations are the same among the Exprs. In an `EXPENSIVE_CHECKS` build, arrays are randomly shuffled prior to sorting them. This exposed the flaky behavior much more often basically breaking the "stability" of the vector - as it should. Because of this, I had to revert the previous fix attempt in llvm#127034. To fix this, I use this time `Expr::getID` for a stable ID for an Expr. Hopefully fixes llvm#126619 Hopefully fixes llvm#126804 (cherry picked from commit f378e52)
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
I have 2 build bots setup to build/test LLVM running on MacOS and I have noticed that the test
clang/test/Analysis/live-stmts.cpprandomly seems to fail on the bot. I have two identically configured workers running the job, and it has failed on both so it doesn't seem specific to one machine configuration.Here is a sample of the failing test output:
Here are some recent runs of the build bot where the test failure occurred:
The text was updated successfully, but these errors were encountered: