Skip to content

[C++20] [Modules] Try to introduce thin BMI to exclude the things not necessary in an interface #71034

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
ChuanqiXu9 opened this issue Nov 2, 2023 · 1 comment · Fixed by #75894
Assignees
Labels
clang:modules C++20 modules and Clang Header Modules

Comments

@ChuanqiXu9
Copy link
Member

ChuanqiXu9 commented Nov 2, 2023

Let's call the current BMI for named modules as fat BMI. Since it contains all the information needed to compile the BMI itself to an object file. And this is the foundation of 2 phase compilations.

However, it is a waste of time and space if we're using one phase compilations.

Then let's call the BMI which doesn't contain all the information to produce an object file, but is able to satisfy the requirement of consumers of the modules as thin BMI. Then it is clear that we should use thin BMI then fat BMI in one phase compilation.

Also in 2 phase compilations, the thin BMI may be helpful too if the build systems want to use a compare-and-swap strategy to generate the BMI based on the BMIDeclsHash (See #71618).

@ChuanqiXu9 ChuanqiXu9 added the clang:modules C++20 modules and Clang Header Modules label Nov 2, 2023
@ChuanqiXu9 ChuanqiXu9 self-assigned this Nov 2, 2023
@llvmbot
Copy link
Member

llvmbot commented Nov 2, 2023

@llvm/issue-subscribers-clang-modules

Author: Chuanqi Xu (ChuanqiXu9)

According to https://isocpp.org/files/papers/P3033R0.html, when we change the body of a non-inline functions in a module purview, the generated BMI shouldn't change.

For example,

export module a;
export int a() {
    return 43;
}

If we change the implementation from return 43; to return 44; the generated BMI shouldn't change. This is pretty helpful to improve the user's experience by avoiding unnecessary rebuildings as much as possible.

This covers: #70569.

Note that the later discussion in SG15 reveals that the paper is not conflicting with 2 phase compilation. Since we can generate 2 BMIs. One for users and one for building self.

ChuanqiXu9 added a commit to ChuanqiXu9/llvm-project that referenced this issue Nov 8, 2023
Close llvm#71034

The patch does following things:
- Introduce thin BMI for C++20 Named modules.
-
@ChuanqiXu9 ChuanqiXu9 changed the title [C++20] [Modules] Try to generate a reduced BMI to exclude the things not necessary in an interface [C++20] [Modules] Try to introduce thin BMI to exclude the things not necessary in an interface Nov 8, 2023
ChuanqiXu9 added a commit to ChuanqiXu9/llvm-project that referenced this issue Nov 8, 2023
Close llvm#71034

This patch introduces thin BMI, which doesn't contain the definitions of
functions and variables if its definitions won't contribute to the ABI.

Testing is a big part of the patch. We want to make sure the thin BMI
contains the same behavior with the existing and relatively stable
fatBMI. This is pretty helpful for further reduction.

For user interfeaces, this patch introduces `-fthinBMI-output=`
arguments to specify the position of thin BMI. This should be used when
compiling a single module unit.

The design is helpful to use thin BMI in two phase compilations too.
With thin BMI, In two phase compilations, we'll generate 2 BMIs, one
thin BMI for being used by consumers, one fat BMI for compiling itself
to object files. Maybe it sounds confusing to have 2 BMIs for one module
unit. But only the thin BMI will be the BMI we're talking about
generally and the fat BMI is only visible by the module unit itself.

With one phase compilation, we may find the behavior of
`-fthinBMI-output=` is pretty similar with `-fmodule-output=`, except
one generating thin BMI and the other generating fat BMI. The design
here is based on 2 things:
(1) The serialization of C++ is pretty complex. We can't be sure we're handling
every detail correctly in the every beginning.
(2) The fat BMI is relatively widely used and relatively stable.
So it looks not good to replace the fat BMI immediately with thin BMI.

But, of course, in the end of the day, we want the consumers to use the
thin BMI only. When that day comes, the `-fmodule-output=` will be an alias to
`-fthinBMI-output=`.

Another design choice is to reuse `-fmodule-output=` and introduce a flag
`-femit-thin-BMI`. Then `-femit-thin-BMI -fmodule-output=` will have the same effect
with `-fthinBMI-output=` now.
The flag `-femit-thin-BMI` should be opt-in now and opt-off later
and finally deprecated.

The roadmap for thin BMI in my mind is:

(1) In clang18, release thin BMI and mark it as experimental. Also
encourage users and build systems to try this new mode.
(2) In clang19 or clang20 (based on the issue feedbacks), remove the
experimental mark for thin BMI and mark fat BMI as deprecated to be
used by consumers.
(3) In clang21 or clang22, error out if we found the users are trying to
import a fat BMI.
ChuanqiXu9 added a commit to ChuanqiXu9/llvm-project that referenced this issue Dec 20, 2023
Close llvm#71034

See
https://discourse.llvm.org/t/rfc-c-20-modules-introduce-thin-bmi-and-decls-hash/74755

This patch introduces reduced BMI, which doesn't contain the definitions of
functions and variables if its definitions won't contribute to the ABI.

Testing is a big part of the patch. We want to make sure the reduced BMI
contains the same behavior with the existing and relatively stable
fatBMI. This is pretty helpful for further reduction.

The user interfaces part it left to following patches to ease the
reviewing.
ChuanqiXu9 added a commit to ChuanqiXu9/llvm-project that referenced this issue Mar 5, 2024
Close llvm#71034

See
https://discourse.llvm.org/t/rfc-c-20-modules-introduce-thin-bmi-and-decls-hash/74755

This patch introduces reduced BMI, which doesn't contain the definitions of
functions and variables if its definitions won't contribute to the ABI.

Testing is a big part of the patch. We want to make sure the reduced BMI
contains the same behavior with the existing and relatively stable
fatBMI. This is pretty helpful for further reduction.

The user interfaces part it left to following patches to ease the
reviewing.
ChuanqiXu9 added a commit to ChuanqiXu9/llvm-project that referenced this issue Mar 8, 2024
Close llvm#71034

See
https://discourse.llvm.org/t/rfc-c-20-modules-introduce-thin-bmi-and-decls-hash/74755

This patch introduces reduced BMI, which doesn't contain the definitions of
functions and variables if its definitions won't contribute to the ABI.

Testing is a big part of the patch. We want to make sure the reduced BMI
contains the same behavior with the existing and relatively stable
fatBMI. This is pretty helpful for further reduction.

The user interfaces part it left to following patches to ease the
reviewing.
ChuanqiXu9 added a commit that referenced this issue Mar 8, 2024
Close #71034

See

https://discourse.llvm.org/t/rfc-c-20-modules-introduce-thin-bmi-and-decls-hash/74755

This patch introduces reduced BMI, which doesn't contain the definitions
of functions and variables if its definitions won't contribute to the
ABI.

Testing is a big part of the patch. We want to make sure the reduced BMI
contains the same behavior with the existing and relatively stable
fatBMI. This is pretty helpful for further reduction.

The user interfaces part it left to following patches to ease the
reviewing.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
clang:modules C++20 modules and Clang Header Modules
Projects
None yet
2 participants