Modules support

[sam-mccall]

It would be nice if some of {C++20 modules, clang header modules} worked in clangd.

Essentially no configurations of this work today: occasionally things may happen to work but there are many reported problems and crashes. These are not just bugs that can be fixed, a design and new infrastructure is needed.

Because clang's module functionality can be enabled by setting driver flags, it is possible/easy for people to end up in this broken and unsupportable state, and wasting (their + our) time debugging it. We should consider failing early and explicitly instead.

AFAIK nobody has plans/availability to work on this soon.

Some issues that need to be addressed:

• which subset of clang's many module features/flags are in scope?
• explicit modules provided by build system: we can't assume these are compatible (may be a different compiler version). So may need to convert to implicitly building modules needed.
• having AST workers/background threads spawning module builds independently and coordinating through the module cache is probably not good, we'll want some kind of worker pool and coordination
• we presumably want to parse modules in the same lite mode as preambles (e.g. skip-function-bodies)
• gcc modules flags are different than clang modules flags (and MSVC presumably different again), we need to parse them
• how do modules fit into the dynamic index? what does a preamble with modules in it look like?
• scattered places where we assume mainfile/preamble dichotomy, reason about #include structure, etc
• actual handling of import/export AST nodes

[HighCommander4]

Some other open bugs we have on file related to modules are #738, #797, #1105.

[Trass3r]

This includes -fmodules? What is the current state?
With an llvm LLVM_ENABLE_LIBCXX=ON LLVM_ENABLE_MODULES=ON workspace clangd doesn't crash or anything but produces lots of squiggles while the build itself runs through just fine.
Shouldn't clangd rather ignore those arguments if it isn't able to handle them properly? (-fmodules -fmodules-cache-path=... -Xclang -fmodules-local-submodule-visibility -gmodules)

[mathstuf]

(Note: speaking as a CMake developer here with a focus on how modules work with CMake; other build systems have similar challenges, but not necessarily the same ones)

gcc modules flags are different than clang modules flags (and MSVC presumably different again), we need to parse them

Note that gcc's module flags may indicate an active participant if -fmodule-mapper= points to a socket or process execution. That may not be emulatable (I believe build2 does this, but CMake uses static files written out during the build).

I suspect that for static analysis, a lot more communication between build systems and tooling to know what is going on. The problem is that with explicit modules even compile_commands.json is not useful until a build happens because:

• for gcc, the -fmodule-mapper= argument is generated at build
• for clang and msvc, modules are communicated via @rspfile arguments that don't exist until the build is performed

This isn't even counting the fact that the information presented there isn't directly usable as any referenced BMI files will be in the host compiler, not something the tool necessarily understands.

In the age we live in, a programming language is only as good as it runtimes/frameworks and lastly editor support... I mean, who codes these days with just syntax highlighting? C++ modules are - from a user perspective - a very nice addition to the C++ syntax end really modernizes it. However, it is really being held back by editor support.

AFAIK: C++ modules are only supported by VisualStudio (using msvc ixx files) and CLion (using clang cppm files). If clangd would support it, it will significantly improve editor support and with that its usability/popularity.

As for compilers having there own flags/options/file-extensions for modules - which is regretful - just ignore all that and start with supporting the clangs own formats.

Although this is - currently - not a critical functionality, I personally am hoping to see modules supported by clangd soon!

[sdykae]

from https://gitlab.kitware.com/cmake/cmake/-/issues/18355

https://github.com/mathstuf/llvm-project/tree/p1689r5-csd
Waiting for stable llvm pr, and support to clangd to start development with vscode extension.

[HighCommander4]

It may be somewhat instructive to look at CLion's approach to supporting modules, which first appeared in the latest 2022.3 release: https://blog.jetbrains.com/clion/2022/11/clion-2022-3-released/#cpp_modules

It's somewhat light on details but this part seems relevant:

CLion collects module information from all .ixx, .cppm, and .mxx files in the project and doesn’t rely on the specific approach you use to build modules. Whether it’s CMake with the Visual Studio C++ toolchain or CMake with Clang and specific compilation flags, CLion provides a similar experience

[ChuanqiXu9]

CLion collects module information from all .ixx, .cppm, and .mxx files in the project and doesn’t rely on the specific approach you use to build modules.

This is not strictly correct since it is allowed to declare a module within a .cpp files. But it should be acceptable as the first step.

It may be somewhat instructive to look at CLion's approach to supporting modules, which first appeared in the latest 2022.3 release: https://blog.jetbrains.com/clion/2022/11/clion-2022-3-released/#cpp_modules

Yeah. Although it didn't mention a lot technical details, I guess we can't learn a lot from this blog. And it only mentioned about the second requirement given by @sam-mccall about the implicit buildings.

---

I guess it may be OK to be too good as the first step. I mean, maybe is it acceptable to ignore some requirements in the first step. Including:

• Compatibility about GCC and MSVC.
• Indexing things.
• lite mode things (e.g. skip-function-bodies)
• #include structures

And what is good as the first step? I think it may be good if:

• We can collect the potential module files by clang-scan-deps in the project tree.
• We can build these module files implicitly in order.
• We can get the AST we just built.

How do you guys think about this? @HighCommander4

[RedBeard0531]

We can collect the potential module files (by its suffixes) in the project tree.

Please don't use extensions to decide which files are modules. It is perfectly valid to have a .cpp file be a module interface. Please just use a scanner. A lot of work went in to the standard wording to ensure that it is possible to very efficiently scan the whole codebase. I know some effort was started in clang-scan-deps for this. It would be great it clangd could reuse and possibly improve this code when doing its scan.