Skip to content

[clangd] "Assertion 'LocalDeclID < LocalNumDecls' failed." when using clangd with --experimental-modules-support #130280

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
jiixyj opened this issue Mar 7, 2025 · 13 comments
Open

[clangd] "Assertion 'LocalDeclID < LocalNumDecls' failed." when using clangd with --experimental-modules-support #130280

jiixyj opened this issue Mar 7, 2025 · 13 comments
Labels
clang:modules C++20 modules and Clang Header Modules clangd crash Prefer [crash-on-valid] or [crash-on-invalid]

Comments

@jiixyj
Copy link
Contributor

jiixyj commented Mar 7, 2025

I'm trying out the new experimental module support of clangd with a ~25 KLOC large code base. On some files (~5 out of ~170), clangd crashes with an assertion failure in clang/lib/Serialization/ASTReader.cpp: "Assertion `LocalDeclID < LocalNumDecls' failed." With assertions disabled there are instead "random" crashes in the parser (I guess because of UB?).

The git hash is ff993f9 .

I haven't found a minimal, standalone reproducer yet, sadly. I have attached the output of clangd --compile-commands-dir=. --experimental-modules-support --check=src/ldgr/core.cppm and the backtrace in gdb.

Any ideas how to further debug this? I haven't found any pattern in the ~5 files that crash, although the crashes are reproducible.

clangd-output.txt
gdb-log.txt
@jiixyj
Copy link
Contributor Author

jiixyj commented Mar 7, 2025

This is the assertion in question:

llvm-project/clang/lib/Serialization/ASTReader.cpp

Line 963 in ff993f9

assert(LocalDeclID < LocalNumDecls);

It isn't hit if I compile the project normally with clang/cmake.

@jiixyj jiixyj changed the title [clangd] "Assertion LocalDeclID < LocalNumDecls' failed." when using clangd with --experimental-modules-support` [clangd] "Assertion 'LocalDeclID < LocalNumDecls' failed." when using clangd with --experimental-modules-support Mar 7, 2025
@frederick-vs-ja frederick-vs-ja added clangd crash Prefer [crash-on-valid] or [crash-on-invalid] and removed new issue labels Mar 7, 2025
@llvmbot
Copy link
Member

llvmbot commented Mar 7, 2025

@llvm/issue-subscribers-clangd

Author: Jan Kokemüller (jiixyj)

I'm trying out the new experimental module support of clangd with a ~25 KLOC large code base. On some files (~5 out of ~170), clangd crashes with an assertion failure in `clang/lib/Serialization/ASTReader.cpp`: "Assertion `LocalDeclID < LocalNumDecls' failed." With assertions disabled there are instead "random" crashes in the parser (I guess because of UB?).

The git hash is ff993f9 .

I haven't found a minimal, standalone reproducer yet, sadly. I have attached the output of clangd --compile-commands-dir=. --experimental-modules-support --check=src/ldgr/core.cppm and the backtrace in gdb.

Any ideas how to further debug this? I haven't found any pattern in the ~5 files that crash, although the crashes are reproducible.

clangd-output.txt
gdb-log.txt

@EugeneZelenko EugeneZelenko added the clang:modules C++20 modules and Clang Header Modules label Mar 7, 2025
@llvmbot
Copy link
Member

llvmbot commented Mar 7, 2025

@llvm/issue-subscribers-clang-modules

Author: Jan Kokemüller (jiixyj)

I'm trying out the new experimental module support of clangd with a ~25 KLOC large code base. On some files (~5 out of ~170), clangd crashes with an assertion failure in `clang/lib/Serialization/ASTReader.cpp`: "Assertion `LocalDeclID < LocalNumDecls' failed." With assertions disabled there are instead "random" crashes in the parser (I guess because of UB?).

The git hash is ff993f9 .

I haven't found a minimal, standalone reproducer yet, sadly. I have attached the output of clangd --compile-commands-dir=. --experimental-modules-support --check=src/ldgr/core.cppm and the backtrace in gdb.

Any ideas how to further debug this? I haven't found any pattern in the ~5 files that crash, although the crashes are reproducible.

clangd-output.txt
gdb-log.txt

@shafik
Copy link
Collaborator

shafik commented Mar 7, 2025

CC @HighCommander4 @ChuanqiXu9

@ChuanqiXu9
Copy link
Member

Did you use the correct compilation databases?

@jiixyj
Copy link
Contributor Author

jiixyj commented Mar 10, 2025

Did you use the correct compilation databases?

I think so. I've symlinked it into the main project folder, but it also crashes if I directly run clangd inside the build folder where the compilation database lives.

I've had a quick look at the output of strace -ttt -f and found that sometimes, the BMI's of the build folder are used, and only sometimes the built ones in /tmp. Could this be a problem?

...
[pid  7100] 1741585644.430668 openat(AT_FDCWD, "/home/jan/git/ldgr/build/RelWithDebInfo/src/CMakeFiles/ldgr.dir/ldgr.util.strong_types.pcm", O_RDONLY|O_CLOEXEC) = 3
[pid  7100] 1741585644.430738 readlink("/proc/self/fd/3", "/home/jan/git/ldgr/build/RelWith"..., 4096) = 90
[pid  7100] 1741585644.430811 fstat(3, {st_mode=S_IFREG|0644, st_size=20976, ...}) = 0
[pid  7100] 1741585644.430897 fstat(3, {st_mode=S_IFREG|0644, st_size=20976, ...}) = 0
[pid  7100] 1741585644.430958 mmap(NULL, 20976, PROT_READ, MAP_PRIVATE|MAP_NORESERVE, 3, 0) = 0x7f828800e000
[pid  7100] 1741585644.431025 rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0
[pid  7100] 1741585644.431081 close(3)  = 0
[pid  7100] 1741585644.431134 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
[pid  7100] 1741585644.431241 openat(AT_FDCWD, "/home/jan/git/ldgr/build/RelWithDebInfo/src/CMakeFiles/ldgr.dir/ldgr.util.utf.pcm", O_RDONLY|O_CLOEXEC) = 3
[pid  7100] 1741585644.431304 readlink("/proc/self/fd/3", "/home/jan/git/ldgr/build/RelWith"..., 4096) = 81
[pid  7100] 1741585644.431369 fstat(3, {st_mode=S_IFREG|0644, st_size=2064280, ...}) = 0
[pid  7100] 1741585644.431460 fstat(3, {st_mode=S_IFREG|0644, st_size=2064280, ...}) = 0
[pid  7100] 1741585644.431517 mmap(NULL, 2064280, PROT_READ, MAP_PRIVATE|MAP_NORESERVE, 3, 0) = 0x7f827ee08000
[pid  7100] 1741585644.431576 rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0
[pid  7100] 1741585644.431633 close(3)  = 0
[pid  7100] 1741585644.431685 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
[pid  7100] 1741585644.453950 newfstatat(AT_FDCWD, "/tmp/clangd/module_files/primitives.cppm9961093566810085741-f2-90-4b-3d-e6-a5", {st_mode=S_IFDIR|0750, st_size=60, ...}, 0) = 0
[pid  7100] 1741585644.454126 openat(AT_FDCWD, "/tmp/clangd/module_files/primitives.cppm9961093566810085741-f2-90-4b-3d-e6-a5/ldgr.primitives.pcm", O_RDONLY|O_CLOEXEC) = 3
[pid  7100] 1741585644.454225 readlink("/proc/self/fd/3", "/tmp/clangd/module_files/primiti"..., 4096) = 97
[pid  7100] 1741585644.454340 fstat(3, {st_mode=S_IFREG|0644, st_size=27556, ...}) = 0
[pid  7100] 1741585644.454432 fstat(3, {st_mode=S_IFREG|0644, st_size=27556, ...}) = 0
[pid  7100] 1741585644.454542 mmap(NULL, 27556, PROT_READ, MAP_PRIVATE|MAP_NORESERVE, 3, 0) = 0x7f8288007000
[pid  7100] 1741585644.454639 rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0
[pid  7100] 1741585644.454730 close(3)  = 0
[pid  7100] 1741585644.454809 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
[pid  7100] 1741585644.454944 openat(AT_FDCWD, "/home/jan/git/ldgr/build/RelWithDebInfo/src/CMakeFiles/ldgr.dir/ldgr.precision.pcm", O_RDONLY|O_CLOEXEC) = 3
[pid  7100] 1741585644.455036 readlink("/proc/self/fd/3", "/home/jan/git/ldgr/build/RelWith"..., 4096) = 82
[pid  7100] 1741585644.455130 fstat(3, {st_mode=S_IFREG|0644, st_size=26680, ...}) = 0
[pid  7100] 1741585644.455216 fstat(3, {st_mode=S_IFREG|0644, st_size=26680, ...}) = 0
[pid  7100] 1741585644.455300 mmap(NULL, 26680, PROT_READ, MAP_PRIVATE|MAP_NORESERVE, 3, 0) = 0x7f8288000000
[pid  7100] 1741585644.455382 rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0
[pid  7100] 1741585644.455470 close(3)  = 0
[pid  7100] 1741585644.455543 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
...

When I delete the .pcm files from the build folder with find src -type f -name '*.pcm' -delete I'm getting a different crash, ending in:

E[07:01:57.080] Failed to build module ldgr.balance_sheet.value; due to Compilation failed
I[07:01:57.080] Built prerequisite modules for file /home/jan/git/ldgr/src/ldgr/report.cpp in 6.72 seconds
I[07:01:57.080] Indexing headers...
I[07:01:57.085] Building AST...
clangd: /freebsd/data/linux/git-net/llvm-project/clang/lib/Tooling/Syntax/Tokens.cpp:717: TokenBuffer clang::syntax::TokenCollector::Builder::build() &&: Assertion `Result.ExpandedTokens.back().kind() == tok::eof' failed.
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.      Program arguments: clangd --compile-commands-dir=. --experimental-modules-support --check=src/ldgr/report.cpp
 #0 0x000055a7499016b8 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) /freebsd/data/linux/git-net/llvm-project/llvm/lib/Support/Unix/Signals.inc:804:13
 #1 0x000055a7498ff900 llvm::sys::RunSignalHandlers() /freebsd/data/linux/git-net/llvm-project/llvm/lib/Support/Signals.cpp:106:18
 #2 0x000055a749901ec1 SignalHandler(int, siginfo_t*, void*) /freebsd/data/linux/git-net/llvm-project/llvm/lib/Support/Unix/Signals.inc:0:3
 #3 0x00007f2c324558c0 __restore_rt libc_sigaction.c:0:0
 #4 0x00007f2c324a462c __pthread_kill_implementation ./nptl/pthread_kill.c:44:76
 #5 0x00007f2c32455822 gsignal ./signal/../sysdeps/posix/raise.c:27:6
 #6 0x00007f2c3243e4af abort ./stdlib/abort.c:81:7
 #7 0x00007f2c3243e3d7 _nl_load_domain.cold ./intl/loadmsgcat.c:1177:9
 #8 0x00007f2c3244e342 (/usr/lib/libc.so.6+0x38342)
 #9 0x000055a74aa2ad0b (/freebsd/data/linux/git-net/llvm-project/prefix/bin/clangd+0x15aad0b)
#10 0x000055a74aa2a802 llvm::DenseMap<clang::SourceLocation, clang::SourceLocation, llvm::DenseMapInfo<clang::SourceLocation, void>, llvm::detail::DenseMapPair<clang::SourceLocation, clang::SourceLocation>>::~DenseMap() /freebsd/data/linux/git-net/llvm-project/llvm/include/llvm/ADT/DenseMap.h:766:23
#11 0x000055a74aa2a802 clang::syntax::TokenCollector::Builder::~Builder() /freebsd/data/linux/git-net/llvm-project/clang/lib/Tooling/Syntax/Tokens.cpp:706:23
#12 0x000055a74aa2a802 clang::syntax::TokenCollector::consume() && /freebsd/data/linux/git-net/llvm-project/clang/lib/Tooling/Syntax/Tokens.cpp:898:3
#13 0x000055a74ac570e2 clang::clangd::ParsedAST::build(llvm::StringRef, clang::clangd::ParseInputs const&, std::unique_ptr<clang::CompilerInvocation, std::default_delete<clang::CompilerInvocation>>, llvm::ArrayRef<clang::clangd::Diag>, std::shared_ptr<clang::clangd::PreambleData const>) /freebsd/data/linux/git-net/llvm-project/clang-tools-extra/clangd/ParsedAST.cpp:726:10
#14 0x000055a74aa3b5bd clang::clangd::(anonymous namespace)::Checker::buildAST() /freebsd/data/linux/git-net/llvm-project/clang-tools-extra/clangd/tool/Check.cpp:262:5
#15 0x000055a74aa3b5bd clang::clangd::check(llvm::StringRef, clang::clangd::ThreadsafeFS const&, clang::clangd::ClangdLSPServer::Options const&) /freebsd/data/linux/git-net/llvm-project/clang-tools-extra/clangd/tool/Check.cpp:522:10
#16 0x000055a74aa30761 clang::clangd::clangdMain(int, char**) /freebsd/data/linux/git-net/llvm-project/clang-tools-extra/clangd/tool/ClangdMain.cpp:1014:12
#17 0x00007f2c3243fc4c __libc_start_call_main ./csu/../sysdeps/nptl/libc_start_call_main.h:74:3
#18 0x00007f2c3243fd05 call_init ./csu/../csu/libc-start.c:128:20
#19 0x00007f2c3243fd05 __libc_start_main@GLIBC_2.2.5 ./csu/../csu/libc-start.c:347:5
#20 0x000055a7497dd9e1 _start /builddir/glibc-2.39/csu/../sysdeps/x86_64/start.S:117:0
zsh: IOT instruction  clangd --compile-commands-dir=. --experimental-modules-support

Maybe those errors are to blame somehow?

error: unable to handle compilation, expected exactly one compiler job in '/freebsd/data/linux/git-net/llvm-project/prefix/bin/clang++ --driver-mode=g++ -I/freebsd/data/linux/git-net/llvm-project/prefix/bin/../lib /x86_64-unknown-linux-gnu/../../share/libc++/v1 -m64 -stdlib=libc++ -fexperimental-library -O2 -g -fcolor-diagnostics -Wno-reserved-module-identifier -c -resource-dir=/freebsd/data/linux/git-net/llvm-project/prefi x/lib/clang/21 -- '

@jiixyj
Copy link
Contributor Author

jiixyj commented Mar 10, 2025

Here is the compilation database in question, generated by CMake 3.30.1.

compile_commands.json

@ChuanqiXu9
Copy link
Member

Did you use the correct compilation databases?

I think so. I've symlinked it into the main project folder, but it also crashes if I directly run clangd inside the build folder where the compilation database lives.

I've had a quick look at the output of strace -ttt -f and found that sometimes, the BMI's of the build folder are used, and only sometimes the built ones in /tmp. Could this be a problem?

I guess so. The assertion says, the loaded decls are more than what expected. And may be it is due to inconsistent setups.

Did you use -fmodules-reduced-bmi in your normal build? If not, I think maybe it worth to give it a try.

...
[pid  7100] 1741585644.430668 openat(AT_FDCWD, "/home/jan/git/ldgr/build/RelWithDebInfo/src/CMakeFiles/ldgr.dir/ldgr.util.strong_types.pcm", O_RDONLY|O_CLOEXEC) = 3
[pid  7100] 1741585644.430738 readlink("/proc/self/fd/3", "/home/jan/git/ldgr/build/RelWith"..., 4096) = 90
[pid  7100] 1741585644.430811 fstat(3, {st_mode=S_IFREG|0644, st_size=20976, ...}) = 0
[pid  7100] 1741585644.430897 fstat(3, {st_mode=S_IFREG|0644, st_size=20976, ...}) = 0
[pid  7100] 1741585644.430958 mmap(NULL, 20976, PROT_READ, MAP_PRIVATE|MAP_NORESERVE, 3, 0) = 0x7f828800e000
[pid  7100] 1741585644.431025 rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0
[pid  7100] 1741585644.431081 close(3)  = 0
[pid  7100] 1741585644.431134 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
[pid  7100] 1741585644.431241 openat(AT_FDCWD, "/home/jan/git/ldgr/build/RelWithDebInfo/src/CMakeFiles/ldgr.dir/ldgr.util.utf.pcm", O_RDONLY|O_CLOEXEC) = 3
[pid  7100] 1741585644.431304 readlink("/proc/self/fd/3", "/home/jan/git/ldgr/build/RelWith"..., 4096) = 81
[pid  7100] 1741585644.431369 fstat(3, {st_mode=S_IFREG|0644, st_size=2064280, ...}) = 0
[pid  7100] 1741585644.431460 fstat(3, {st_mode=S_IFREG|0644, st_size=2064280, ...}) = 0
[pid  7100] 1741585644.431517 mmap(NULL, 2064280, PROT_READ, MAP_PRIVATE|MAP_NORESERVE, 3, 0) = 0x7f827ee08000
[pid  7100] 1741585644.431576 rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0
[pid  7100] 1741585644.431633 close(3)  = 0
[pid  7100] 1741585644.431685 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
[pid  7100] 1741585644.453950 newfstatat(AT_FDCWD, "/tmp/clangd/module_files/primitives.cppm9961093566810085741-f2-90-4b-3d-e6-a5", {st_mode=S_IFDIR|0750, st_size=60, ...}, 0) = 0
[pid  7100] 1741585644.454126 openat(AT_FDCWD, "/tmp/clangd/module_files/primitives.cppm9961093566810085741-f2-90-4b-3d-e6-a5/ldgr.primitives.pcm", O_RDONLY|O_CLOEXEC) = 3
[pid  7100] 1741585644.454225 readlink("/proc/self/fd/3", "/tmp/clangd/module_files/primiti"..., 4096) = 97
[pid  7100] 1741585644.454340 fstat(3, {st_mode=S_IFREG|0644, st_size=27556, ...}) = 0
[pid  7100] 1741585644.454432 fstat(3, {st_mode=S_IFREG|0644, st_size=27556, ...}) = 0
[pid  7100] 1741585644.454542 mmap(NULL, 27556, PROT_READ, MAP_PRIVATE|MAP_NORESERVE, 3, 0) = 0x7f8288007000
[pid  7100] 1741585644.454639 rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0
[pid  7100] 1741585644.454730 close(3)  = 0
[pid  7100] 1741585644.454809 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
[pid  7100] 1741585644.454944 openat(AT_FDCWD, "/home/jan/git/ldgr/build/RelWithDebInfo/src/CMakeFiles/ldgr.dir/ldgr.precision.pcm", O_RDONLY|O_CLOEXEC) = 3
[pid  7100] 1741585644.455036 readlink("/proc/self/fd/3", "/home/jan/git/ldgr/build/RelWith"..., 4096) = 82
[pid  7100] 1741585644.455130 fstat(3, {st_mode=S_IFREG|0644, st_size=26680, ...}) = 0
[pid  7100] 1741585644.455216 fstat(3, {st_mode=S_IFREG|0644, st_size=26680, ...}) = 0
[pid  7100] 1741585644.455300 mmap(NULL, 26680, PROT_READ, MAP_PRIVATE|MAP_NORESERVE, 3, 0) = 0x7f8288000000
[pid  7100] 1741585644.455382 rt_sigprocmask(SIG_SETMASK, ~[RTMIN RT_1], [], 8) = 0
[pid  7100] 1741585644.455470 close(3)  = 0
[pid  7100] 1741585644.455543 rt_sigprocmask(SIG_SETMASK, [], NULL, 8) = 0
...

When I delete the .pcm files from the build folder with find src -type f -name '*.pcm' -delete I'm getting a different crash, ending in:

E[07:01:57.080] Failed to build module ldgr.balance_sheet.value; due to Compilation failed
I[07:01:57.080] Built prerequisite modules for file /home/jan/git/ldgr/src/ldgr/report.cpp in 6.72 seconds
I[07:01:57.080] Indexing headers...
I[07:01:57.085] Building AST...
clangd: /freebsd/data/linux/git-net/llvm-project/clang/lib/Tooling/Syntax/Tokens.cpp:717: TokenBuffer clang::syntax::TokenCollector::Builder::build() &&: Assertion `Result.ExpandedTokens.back().kind() == tok::eof' failed.
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.
Stack dump:
0.      Program arguments: clangd --compile-commands-dir=. --experimental-modules-support --check=src/ldgr/report.cpp
 #0 0x000055a7499016b8 llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) /freebsd/data/linux/git-net/llvm-project/llvm/lib/Support/Unix/Signals.inc:804:13
 #1 0x000055a7498ff900 llvm::sys::RunSignalHandlers() /freebsd/data/linux/git-net/llvm-project/llvm/lib/Support/Signals.cpp:106:18
 #2 0x000055a749901ec1 SignalHandler(int, siginfo_t*, void*) /freebsd/data/linux/git-net/llvm-project/llvm/lib/Support/Unix/Signals.inc:0:3
 #3 0x00007f2c324558c0 __restore_rt libc_sigaction.c:0:0
 #4 0x00007f2c324a462c __pthread_kill_implementation ./nptl/pthread_kill.c:44:76
 #5 0x00007f2c32455822 gsignal ./signal/../sysdeps/posix/raise.c:27:6
 #6 0x00007f2c3243e4af abort ./stdlib/abort.c:81:7
 #7 0x00007f2c3243e3d7 _nl_load_domain.cold ./intl/loadmsgcat.c:1177:9
 #8 0x00007f2c3244e342 (/usr/lib/libc.so.6+0x38342)
 #9 0x000055a74aa2ad0b (/freebsd/data/linux/git-net/llvm-project/prefix/bin/clangd+0x15aad0b)
#10 0x000055a74aa2a802 llvm::DenseMap<clang::SourceLocation, clang::SourceLocation, llvm::DenseMapInfo<clang::SourceLocation, void>, llvm::detail::DenseMapPair<clang::SourceLocation, clang::SourceLocation>>::~DenseMap() /freebsd/data/linux/git-net/llvm-project/llvm/include/llvm/ADT/DenseMap.h:766:23
#11 0x000055a74aa2a802 clang::syntax::TokenCollector::Builder::~Builder() /freebsd/data/linux/git-net/llvm-project/clang/lib/Tooling/Syntax/Tokens.cpp:706:23
#12 0x000055a74aa2a802 clang::syntax::TokenCollector::consume() && /freebsd/data/linux/git-net/llvm-project/clang/lib/Tooling/Syntax/Tokens.cpp:898:3
#13 0x000055a74ac570e2 clang::clangd::ParsedAST::build(llvm::StringRef, clang::clangd::ParseInputs const&, std::unique_ptr<clang::CompilerInvocation, std::default_delete<clang::CompilerInvocation>>, llvm::ArrayRef<clang::clangd::Diag>, std::shared_ptr<clang::clangd::PreambleData const>) /freebsd/data/linux/git-net/llvm-project/clang-tools-extra/clangd/ParsedAST.cpp:726:10
#14 0x000055a74aa3b5bd clang::clangd::(anonymous namespace)::Checker::buildAST() /freebsd/data/linux/git-net/llvm-project/clang-tools-extra/clangd/tool/Check.cpp:262:5
#15 0x000055a74aa3b5bd clang::clangd::check(llvm::StringRef, clang::clangd::ThreadsafeFS const&, clang::clangd::ClangdLSPServer::Options const&) /freebsd/data/linux/git-net/llvm-project/clang-tools-extra/clangd/tool/Check.cpp:522:10
#16 0x000055a74aa30761 clang::clangd::clangdMain(int, char**) /freebsd/data/linux/git-net/llvm-project/clang-tools-extra/clangd/tool/ClangdMain.cpp:1014:12
#17 0x00007f2c3243fc4c __libc_start_call_main ./csu/../sysdeps/nptl/libc_start_call_main.h:74:3
#18 0x00007f2c3243fd05 call_init ./csu/../csu/libc-start.c:128:20
#19 0x00007f2c3243fd05 __libc_start_main@GLIBC_2.2.5 ./csu/../csu/libc-start.c:347:5
#20 0x000055a7497dd9e1 _start /builddir/glibc-2.39/csu/../sysdeps/x86_64/start.S:117:0
zsh: IOT instruction  clangd --compile-commands-dir=. --experimental-modules-support

Maybe those errors are to blame somehow?

error: unable to handle compilation, expected exactly one compiler job in '/freebsd/data/linux/git-net/llvm-project/prefix/bin/clang++ --driver-mode=g++ -I/freebsd/data/linux/git-net/llvm-project/prefix/bin/../lib /x86_64-unknown-linux-gnu/../../share/libc++/v1 -m64 -stdlib=libc++ -fexperimental-library -O2 -g -fcolor-diagnostics -Wno-reserved-module-identifier -c -resource-dir=/freebsd/data/linux/git-net/llvm-project/prefi x/lib/clang/21 -- '

I had no clue. Did you restart the clangd server after removing these .pcm in build dir?

@ChuanqiXu9
Copy link
Member

Here is the compilation database in question, generated by CMake 3.30.1.

compile_commands.json

Oh, it looks like CMake generate the compilation database with different manner with previous version, which I tested with.

Could you try again with removing all pattern like @[^\s\S]+.modmap?

@jiixyj
Copy link
Contributor Author

jiixyj commented Mar 10, 2025

Did you use -fmodules-reduced-bmi in your normal build? If not, I think maybe it worth to give it a try.

OK, I will try this!

I had no clue. Did you restart the clangd server after removing these .pcm in build dir?

Yes, to get those outputs I'm running clangd in --check= mode.

Could you try again with removing all pattern like @[^\s\S]+.modmap?

I'll try this as well.

@jiixyj
Copy link
Contributor Author

jiixyj commented Mar 10, 2025

Oh, it looks like CMake generate the compilation database with different manner with previous version, which I tested with.

Just as another data point, those .modmap files look approximately like this:

-x c++-module
-fmodule-output=src/CMakeFiles/ldgr.test.dir/ldgr.number.test.pcm
-fmodule-file=std=CMakeFiles/__cmake_cxx26.dir/std.pcm
-fmodule-file=ldgr.base=src/CMakeFiles/ldgr.dir/ldgr.base.pcm
-fmodule-file=ldgr.number.parse=src/CMakeFiles/ldgr.dir/ldgr.number.parse.pcm
-fmodule-file=bopt=src/CMakeFiles/bopt__bopt@synth_a03eb766914c.dir/bc55359fac96.bmi
-fmodule-file=bopt.datasizeof=src/CMakeFiles/bopt__bopt@synth_a03eb766914c.dir/d3c749a0b229.bmi
-fmodule-file=bopt.invoke_tag=src/CMakeFiles/bopt__bopt@synth_a03eb766914c.dir/08a59c252756.bmi
-fmodule-file=bopt.niche=src/CMakeFiles/bopt__bopt@synth_a03eb766914c.dir/43774037a3a1.bmi
-fmodule-file=bopt.optional=src/CMakeFiles/bopt__bopt@synth_a03eb766914c.dir/b1a0f9067bb9.bmi
-fmodule-file=erased=src/CMakeFiles/erased__erased@synth_a03eb766914c.dir/1869a037957f.bmi
-fmodule-file=erased.any_with_interfaces=src/CMakeFiles/erased__erased@synth_a03eb766914c.dir/6c4f8b6c84f3.bmi
-fmodule-file=erased.basic_cpo=src/CMakeFiles/erased__erased@synth_a03eb766914c.dir/67515b8d845e.bmi
-fmodule-file=erased.std_cpo=src/CMakeFiles/erased__erased@synth_a03eb766914c.dir/3474a11866c5.bmi
-fmodule-file=erased.type_list=src/CMakeFiles/erased__erased@synth_a03eb766914c.dir/48c3ccda53d6.bmi
-fmodule-file=erased.util=src/CMakeFiles/erased__erased@synth_a03eb766914c.dir/39d4002825dd.bmi
-fmodule-file=erased.vtable=src/CMakeFiles/erased__erased@synth_a03eb766914c.dir/28aafd22888b.bmi
-fmodule-file=erased.wrappers=src/CMakeFiles/erased__erased@synth_a03eb766914c.dir/05c6b00a7d45.bmi
-fmodule-file=ldgr.error=src/CMakeFiles/ldgr.dir/ldgr.error.pcm
-fmodule-file=ldgr.number=src/CMakeFiles/ldgr.dir/ldgr.number.pcm
-fmodule-file=ldgr.precision=src/CMakeFiles/ldgr.dir/ldgr.precision.pcm
-fmodule-file=ldgr.util=src/CMakeFiles/ldgr.dir/ldgr.util.pcm
-fmodule-file=ldgr.util.any_cast=src/CMakeFiles/ldgr.dir/ldgr.util.any_cast.pcm
-fmodule-file=ldgr.util.any_input_iterator=src/CMakeFiles/ldgr.dir/ldgr.util.any_input_iterator.pcm
-fmodule-file=ldgr.util.casts=src/CMakeFiles/ldgr.dir/ldgr.util.casts.pcm
...

@naveen-seth
Copy link
Contributor

naveen-seth commented Apr 11, 2025 •
edited
Loading

I've had a quick look at the output of strace -ttt -f and found that sometimes, the BMI's of the build folder are used, and only sometimes the built ones in /tmp. Could this be a problem?

The execution trace in gdb-log.txt make me believe that the crash itself is related to issue #132059 and should be fixed in trunk by #133462.
The inconsistent use of /tmp and the build directory (or some other inconsistency in the setup) will probably still cause the compilation to fail (without crashing) and could be a different bug.

Using a clangd build from the given git hash, I was able to make clangd crash by creating a CMake build for the test introduced by #133462 and modifying CMakeFiles/maindir/main.cpp.o.modmap to provide the same faulty -fmodule-file mapping as in the test.

Crash 
nav@nixos:~/llvm-project/debug-env/test1$ ../../build-ff993f9/bin/clangd --compile-commands-dir=./ff993f9-build --check=main.cpp  
I[15:35:14.316] clangd version 21.0.0git ([email protected]:llvm/llvm-project.git ff993f9a4c23cfaae54f2a2708bed23954b8d9c0)
I[15:35:14.316] Features: linux+debug
I[15:35:14.316] PID: 138061
I[15:35:14.316] Working directory: /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/debug-env/test1
I[15:35:14.316] argv[0]: ../../build-ff993f9/bin/clangd
I[15:35:14.316] argv[1]: --compile-commands-dir=./ff993f9-build
I[15:35:14.316] argv[2]: --check=main.cpp
I[15:35:14.316] Entering check mode (no LSP server)
I[15:35:14.316] Testing on source file /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/debug-env/test1/main.cpp
I[15:35:14.316] Loading compilation database...
I[15:35:14.317] Loaded compilation database from /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/debug-env/test1/./ff993f9-build/compile_commands.json
I[15:35:14.318] Compile command from CDB is: [/home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/debug-env/test1/ff993f9-build] /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/build-ff993f9/bin/clang++ --driver-mode=g++ -std=gnu++23 -fmodule-file=A=CMakeFiles/main.dir/B.pcm -o CMakeFiles/main.dir/main.cpp.o -c -resource-dir=/home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/build-ff993f9/lib/clang/21 -- /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/debug-env/test1/main.cpp
I[15:35:14.319] Parsing command...
I[15:35:14.321] internal (cc1) args are: -cc1 -triple x86_64-unknown-linux-gnu -fsyntax-only -disable-free -clear-ast-before-backend -main-file-name main.cpp -mrelocation-model pic -pic-level 2 -pic-is-pie -mframe-pointer=all -fmath-errno -ffp-contract=on -fno-rounding-math -mconstructor-aliases -funwind-tables=2 -target-cpu x86-64 -tune-cpu generic -debugger-tuning=gdb -fdebug-compilation-dir=/home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/debug-env/test1/ff993f9-build -fcoverage-compilation-dir=/home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/debug-env/test1/ff993f9-build -resource-dir /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/build-ff993f9/lib/clang/21 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/14/../../../../include/c++/14 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/14/../../../../include/x86_64-linux-gnu/c++/14 -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/14/../../../../include/c++/14/backward -internal-isystem /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/build-ff993f9/lib/clang/21/include -internal-isystem /usr/local/include -internal-isystem /usr/lib/gcc/x86_64-linux-gnu/14/../../../../x86_64-linux-gnu/include -internal-externc-isystem /usr/include/x86_64-linux-gnu -internal-externc-isystem /include -internal-externc-isystem /usr/include -std=gnu++23 -fdeprecated-macro -ferror-limit 19 -fgnuc-version=4.2.1 -fno-implicit-modules -fmodule-file=A=CMakeFiles/main.dir/B.pcm -fskip-odr-check-in-gmf -fcxx-exceptions -fexceptions -no-round-trip-args -faddrsig -D__GCC_HAVE_DWARF2_CFI_ASM=1 -x c++ /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/debug-env/test1/main.cpp
I[15:35:14.322] Building preamble...
I[15:35:14.347] Built preamble of size 261968 for file /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/debug-env/test1/main.cpp version null in 0.02 seconds
I[15:35:14.347] Indexing headers...
I[15:35:14.349] Building AST...
clangd: /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/clang/lib/Serialization/ASTReader.cpp:9581: serialization::SubmoduleID clang::ASTReader::getGlobalSubmoduleID(ModuleFile &, unsigned int) const: Assertion `I != M.SubmoduleRemap.end() && "Invalid index into submodule index remap"' failed.
PLEASE submit a bug report to https://github.com/llvm/llvm-project/issues/ and include the crash backtrace.Stack dump:
0.      Program arguments: ../../build-ff993f9/bin/clangd --compile-commands-dir=./ff993f9-build --check=main.cpp
1.      /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/debug-env/test1/main.cpp:4:1: current parser token 'int'
 #0 0x000055f1e1a2471d llvm::sys::PrintStackTrace(llvm::raw_ostream&, int) /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/llvm/lib/Support/Unix/Signals.inc:804:11
 #1 0x000055f1e1a24bdb PrintStackTraceSignalHandler(void*) /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/llvm/lib/Support/Unix/Signals.inc:880:1
 #2 0x000055f1e1a22d7f llvm::sys::RunSignalHandlers() /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/llvm/lib/Support/Signals.cpp:105:5
 #3 0x000055f1e1a252b9 SignalHandler(int, siginfo_t*, void*) /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/llvm/lib/Support/Unix/Signals.inc:418:7
 #4 0x00007f86ac59c250 (/lib/x86_64-linux-gnu/libc.so.6+0x45250)
 #5 0x00007f86ac5faf1c pthread_kill (/lib/x86_64-linux-gnu/libc.so.6+0xa3f1c)
 #6 0x00007f86ac59c19e raise (/lib/x86_64-linux-gnu/libc.so.6+0x4519e)
 #7 0x00007f86ac57f902 abort (/lib/x86_64-linux-gnu/libc.so.6+0x28902)
 #8 0x00007f86ac57f81e (/lib/x86_64-linux-gnu/libc.so.6+0x2881e)
 #9 0x00007f86ac5927c7 (/lib/x86_64-linux-gnu/libc.so.6+0x3b7c7)
#10 0x000055f1e651b981 clang::ASTReader::getGlobalSubmoduleID(clang::serialization::ModuleFile&, unsigned int) const /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/clang/lib/Serialization/ASTReader.cpp:9583:10
#11 0x000055f1e6531c29 clang::ASTReader::ReadAST(llvm::StringRef, clang::serialization::ModuleKind, clang::SourceLocation, unsigned int, clang::serialization::ModuleFile**) /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/clang/lib/Serialization/ASTReader.cpp:4834:28
#12 0x000055f1e60aba14 clang::CompilerInstance::findOrCompileModuleAndReadAST(llvm::StringRef, clang::SourceLocation, clang::SourceLocation, bool) /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/clang/lib/Frontend/CompilerInstance.cpp:1902:27
#13 0x000055f1e60ac960 clang::CompilerInstance::loadModule(clang::SourceLocation, llvm::ArrayRef<std::pair<clang::IdentifierInfo*, clang::SourceLocation>>, clang::Module::NameVisibilityKind, bool) /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/clang/lib/Frontend/CompilerInstance.cpp:2054:31
#14 0x000055f1e335c660 clang::Sema::ActOnModuleImport(clang::SourceLocation, clang::SourceLocation, clang::SourceLocation, llvm::ArrayRef<std::pair<clang::IdentifierInfo*, clang::SourceLocation>>, bool) /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/clang/lib/Sema/SemaModule.cpp:616:35
#15 0x000055f1e64ecec7 clang::Parser::ParseModuleImport(clang::SourceLocation, clang::Sema::ModuleImportState&) /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/clang/lib/Parse/Parser.cpp:2667:22
#16 0x000055f1e64eba6d clang::Parser::ParseTopLevelDecl(clang::OpaquePtr<clang::DeclGroupRef>&, clang::Sema::ModuleImportState&) /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/clang/lib/Parse/Parser.cpp:673:24
#17 0x000055f1e64eb7a0 clang::Parser::ParseFirstTopLevelDecl(clang::OpaquePtr<clang::DeclGroupRef>&, clang::Sema::ModuleImportState&) /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/clang/lib/Parse/Parser.cpp:608:8
#18 0x000055f1e63e16b9 clang::ParseAST(clang::Sema&, bool, bool) /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/clang/lib/Parse/ParseAST.cpp:170:15
#19 0x000055f1e617ab47 clang::ASTFrontendAction::ExecuteAction() /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/clang/lib/Frontend/FrontendAction.cpp:1188:1
#20 0x000055f1e617a5a6 clang::FrontendAction::Execute() /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/clang/lib/Frontend/FrontendAction.cpp:1076:7
#21 0x000055f1e40eaf97 clang::clangd::ParsedAST::build(llvm::StringRef, clang::clangd::ParseInputs const&, std::unique_ptr<clang::CompilerInvocation, std::default_delete<clang::CompilerInvocation>>, llvm::ArrayRef<clang::clangd::Diag>, std::shared_ptr<clang::clangd::PreambleData const>) /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/clang-tools-extra/clangd/ParsedAST.cpp:713:19
#22 0x000055f1e3bd33a6 clang::clangd::(anonymous namespace)::Checker::buildAST() /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/clang-tools-extra/clangd/tool/Check.cpp:262:11
#23 0x000055f1e3bd249c clang::clangd::check(llvm::StringRef, clang::clangd::ThreadsafeFS const&, clang::clangd::ClangdLSPServer::Options const&) /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/clang-tools-extra/clangd/tool/Check.cpp:522:7
#24 0x000055f1e3ba0704 clang::clangd::clangdMain(int, char**) /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/clang-tools-extra/clangd/tool/ClangdMain.cpp:1014:12
#25 0x000055f1e186cf92 main /home/nav/.distrobox/ubuntu-24-10-compiler-dev/llvm-project/clang-tools-extra/clangd/tool/ClangdToolMain.cpp:12:3
#26 0x00007f86ac5813b8 (/lib/x86_64-linux-gnu/libc.so.6+0x2a3b8)
#27 0x00007f86ac58147b __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x2a47b)
#28 0x000055f1e15208e5 _start (../../build-ff993f9/bin/clangd+0x33d68e5)
Aborted (core dumped)

Because exact location of the crash depends on the source file contents (context), I couldn’t reproduce the crash on the exactly same assert, but also from inside of clang::ASTReader::ReadDeclID.

When I delete the .pcm files from the build folder with find src -type f -name '*.pcm' -delete I'm getting a different crash, ending in: ...

(For the second crash, I am clueless.)

@ChuanqiXu9
Copy link
Member

I've had a quick look at the output of strace -ttt -f and found that sometimes, the BMI's of the build folder are used, and only sometimes the built ones in /tmp. Could this be a problem?

The execution trace in gdb-log.txt make me believe that the crash itself is related to issue #132059 and should be fixed in trunk by #133462. The inconsistent use of /tmp and the build directory (or some other inconsistency in the setup) will probably still cause the compilation to fail (without crashing) and could be a different bug.

Using a clangd build from the given git hash, I was able to make clangd crash by creating a CMake build for the test introduced by #133462 and modifying CMakeFiles/maindir/main.cpp.o.modmap to provide the same faulty -fmodule-file mapping as in the test.

Crash
Because exact location of the crash depends on the source file contents (context), I couldn’t reproduce the crash on the exactly same assert, but also from inside of clang::ASTReader::ReadDeclID.

When I delete the .pcm files from the build folder with find src -type f -name '*.pcm' -delete I'm getting a different crash, ending in: ...

(For the second crash, I am clueless.)

Thanks for looking into this. Do the crash still exist if the PCMs in the build directory are not used?

We intended to ignore all PCMs built in build dir in clang-tools-extra/clangd/ModulesBuilder.cpp:155 : adjustHeaderSearchOptions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
clang:modules C++20 modules and Clang Header Modules clangd crash Prefer [crash-on-valid] or [crash-on-invalid]
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
7 participants
@jiixyj @shafik @EugeneZelenko @frederick-vs-ja @llvmbot @ChuanqiXu9 @naveen-seth