[v30.0-rc2 regression] `src/google/protobuf/compiler/java/context.h` causes compile error C2079 in MSVC with `/std:c++20` and `/Zc:__cplusplus` #20331

yukawa · 2025-02-12T10:35:47Z

What version of protobuf and what language are you using?
Version: 40ec76e
Language: C++

What operating system (Linux, Windows, ...) and version?
Windows 11 24H2

What runtime / compiler are you using (e.g., python version or gcc version)
Microsoft (R) C/C++ Optimizing Compiler Version 19.42.34438 for x64

What did you do?
Steps to reproduce the behavior:

git clone https://github.com/protocolbuffers/protobuf.git
cd protobuf
git checkout 40ec76ef674edb4cca75411d26b1984a8bf1eab6
cd examples
bazelisk --output_base=C:/x --ignore_all_rc_files --output_base=C:/x build addressbook_cc_proto --define=protobuf_allow_msvc=true --cxxopt "/std:c++20" --host_cxxopt "/std:c++20" --cxxopt "/Zc:__cplusplus" --host_cxxopt "/Zc:__cplusplus" --verbose_failures

What did you expect to see
The build succeeds.

What did you see instead?

INFO: Analyzed target //:addressbook_cc_proto (0 packages loaded, 0 targets configured).
ERROR: C:/x/external/protobuf~/src/google/protobuf/compiler/java/BUILD.bazel:68:11: Compiling src/google/protobuf/compiler/java/internal_helpers.cc [for tool] failed: (Exit 2): cl.exe failed: error executing CppCompile command (from target @@protobuf~//src/google/protobuf/compiler/java:internal_helpers)
  cd /d C:/x/execroot/_main
  SET INCLUDE=C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.42.34433\include;C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.42.34433\ATLMFC\include;C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\VS\include;C:\Program Files (x86)\Windows Kits\10\include\10.0.26100.0\ucrt;C:\Program Files (x86)\Windows Kits\10\\include\10.0.26100.0\\um;C:\Program Files (x86)\Windows Kits\10\\include\10.0.26100.0\\shared;C:\Program Files (x86)\Windows Kits\10\\include\10.0.26100.0\\winrt;C:\Program Files (x86)\Windows Kits\10\\include\10.0.26100.0\\cppwinrt;C:\Program Files (x86)\Windows Kits\NETFXSDK\4.8\include\um
    SET PATH=C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.42.34433\bin\HostX64\x64;C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\VC\VCPackages;C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\TestWindow;C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\TeamFoundation\Team Explorer;C:\Program Files\Microsoft Visual Studio\2022\Community\MSBuild\Current\bin\Roslyn;C:\Program Files (x86)\Microsoft SDKs\Windows\v10.0A\bin\NETFX 4.8 Tools\x64\;C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\FSharp\Tools;C:\Program Files\Microsoft Visual Studio\2022\Community\Team Tools\DiagnosticsHub\Collector;C:\Program Files (x86)\Windows Kits\10\bin\10.0.26100.0\\x64;C:\Program Files (x86)\Windows Kits\10\bin\\x64;C:\Program Files\Microsoft Visual Studio\2022\Community\\MSBuild\Current\Bin\amd64;C:\Windows\Microsoft.NET\Framework64\v4.0.30319;C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\;C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\Tools\;;C:\WINDOWS\system32;C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\CMake\CMake\bin;C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\CommonExtensions\Microsoft\CMake\Ninja;C:\Program Files\Microsoft Visual Studio\2022\Community\Common7\IDE\VC\Linux\bin\ConnectionManagerExe;C:\Program Files\Microsoft Visual Studio\2022\Community\VC\vcpkg
    SET PWD=/proc/self/cwd
    SET RUNFILES_MANIFEST_ONLY=1
    SET TEMP=C:\Users\yukawa\AppData\Local\Temp
    SET TMP=C:\Users\yukawa\AppData\Local\Temp
    SET VSLANG=1033
  C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.42.34433\bin\HostX64\x64\cl.exe @bazel-out/x64_windows-opt-exec-ST-13d3ddad9198/bin/external/protobuf~/src/google/protobuf/compiler/java/_objs/internal_helpers/internal_helpers.obj.params
# Configuration: f7e5f1179875076cb7b4f3ddd6965b5716e54d99ae929d3d122a387b659f99f3
# Execution platform: @@local_config_platform//:host
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.42.34433\include\utility(470): error C2079: 'std::pair<const google::protobuf::FieldDescriptor *,google::protobuf::compiler::java::FieldGeneratorInfo>::second' uses undefined struct 'google::protobuf::compiler::java::FieldGeneratorInfo'
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.42.34433\include\utility(470): note: the template instantiation context (the oldest one first) is
bazel-out/x64_windows-opt-exec-ST-13d3ddad9198/bin/external/protobuf~/src/google/protobuf/compiler/java/_virtual_includes/helpers\google/protobuf/compiler/java/context.h(81): note: see reference to class template instantiation 'absl::lts_20250127::flat_hash_map<const google::protobuf::FieldDescriptor *,google::protobuf::compiler::java::FieldGeneratorInfo,absl::lts_20250127::container_internal::HashEq<K,void>::Hash,absl::lts_20250127::container_internal::HashEq<K,void>::Eq,std::allocator<std::pair<const K,V>>>' being compiled
        with
        [
            K=const google::protobuf::FieldDescriptor *,
            V=google::protobuf::compiler::java::FieldGeneratorInfo
        ]
external/abseil-cpp~\absl/container/flat_hash_map.h(129): note: see reference to class template instantiation 'absl::lts_20250127::container_internal::raw_hash_map<absl::lts_20250127::container_internal::FlatHashMapPolicy<K,V>,Hash,Eq,Allocator>' being compiled
        with
        [
            K=const google::protobuf::FieldDescriptor *,
            V=google::protobuf::compiler::java::FieldGeneratorInfo,
            Hash=absl::lts_20250127::container_internal::HashEq<const google::protobuf::FieldDescriptor *,void>::Hash,
            Eq=absl::lts_20250127::container_internal::HashEq<const google::protobuf::FieldDescriptor *,void>::Eq,
            Allocator=std::allocator<std::pair<const google::protobuf::FieldDescriptor *const ,google::protobuf::compiler::java::FieldGeneratorInfo>>
        ]
external/abseil-cpp~\absl/container/internal/raw_hash_map.h(33): note: see reference to class template instantiation 'absl::lts_20250127::container_internal::raw_hash_set<Policy,Hash,Eq,Alloc>' being compiled
        with
        [
            Policy=absl::lts_20250127::container_internal::FlatHashMapPolicy<const google::protobuf::FieldDescriptor *,google::protobuf::compiler::java::FieldGeneratorInfo>,
            Hash=absl::lts_20250127::container_internal::HashEq<const google::protobuf::FieldDescriptor *,void>::Hash,
            Eq=absl::lts_20250127::container_internal::HashEq<const google::protobuf::FieldDescriptor *,void>::Eq,
            Alloc=std::allocator<std::pair<const google::protobuf::FieldDescriptor *const ,google::protobuf::compiler::java::FieldGeneratorInfo>>
        ]
external/abseil-cpp~\absl/container/internal/raw_hash_set.h(3030): note: see reference to class template instantiation 'absl::lts_20250127::conjunction<std::integral_constant<bool,true>,absl::lts_20250127::type_traits_internal::IsOwner<std::pair<const google::protobuf::FieldDescriptor *,google::protobuf::compiler::java::FieldGeneratorInfo>>,absl::lts_20250127::type_traits_internal::IsView<std::pair<const google::protobuf::FieldDescriptor *,google::protobuf::compiler::java::FieldGeneratorInfo>>>' being compiled
external/abseil-cpp~\absl/meta/type_traits.h(147): note: see reference to class template instantiation 'absl::lts_20250127::conjunction<absl::lts_20250127::type_traits_internal::IsOwner<std::pair<const google::protobuf::FieldDescriptor *,google::protobuf::compiler::java::FieldGeneratorInfo>>,absl::lts_20250127::type_traits_internal::IsView<std::pair<const google::protobuf::FieldDescriptor *,google::protobuf::compiler::java::FieldGeneratorInfo>>>' being compiled
external/abseil-cpp~\absl/meta/type_traits.h(147): note: see reference to class template instantiation 'absl::lts_20250127::type_traits_internal::IsOwner<std::pair<const google::protobuf::FieldDescriptor *,google::protobuf::compiler::java::FieldGeneratorInfo>>' being compiled
external/abseil-cpp~\absl/meta/type_traits.h(607): note: see reference to class template instantiation 'std::pair<const google::protobuf::FieldDescriptor *,google::protobuf::compiler::java::FieldGeneratorInfo>' being compiled
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.42.34433\include\utility(470): error C2079: 'std::pair<const google::protobuf::OneofDescriptor *,google::protobuf::compiler::java::OneofGeneratorInfo>::second' uses undefined struct 'google::protobuf::compiler::java::OneofGeneratorInfo'
C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Tools\MSVC\14.42.34433\include\utility(470): note: the template instantiation context (the oldest one first) is
bazel-out/x64_windows-opt-exec-ST-13d3ddad9198/bin/external/protobuf~/src/google/protobuf/compiler/java/_virtual_includes/helpers\google/protobuf/compiler/java/context.h(83): note: see reference to class template instantiation 'absl::lts_20250127::flat_hash_map<const google::protobuf::OneofDescriptor *,google::protobuf::compiler::java::OneofGeneratorInfo,absl::lts_20250127::container_internal::HashEq<K,void>::Hash,absl::lts_20250127::container_internal::HashEq<K,void>::Eq,std::allocator<std::pair<const K,V>>>' being compiled
        with
        [
            K=const google::protobuf::OneofDescriptor *,
            V=google::protobuf::compiler::java::OneofGeneratorInfo
        ]
external/abseil-cpp~\absl/container/flat_hash_map.h(129): note: see reference to class template instantiation 'absl::lts_20250127::container_internal::raw_hash_map<absl::lts_20250127::container_internal::FlatHashMapPolicy<K,V>,Hash,Eq,Allocator>' being compiled
        with
        [
            K=const google::protobuf::OneofDescriptor *,
            V=google::protobuf::compiler::java::OneofGeneratorInfo,
            Hash=absl::lts_20250127::container_internal::HashEq<const google::protobuf::OneofDescriptor *,void>::Hash,
            Eq=absl::lts_20250127::container_internal::HashEq<const google::protobuf::OneofDescriptor *,void>::Eq,
            Allocator=std::allocator<std::pair<const google::protobuf::OneofDescriptor *const ,google::protobuf::compiler::java::OneofGeneratorInfo>>
        ]
external/abseil-cpp~\absl/container/internal/raw_hash_map.h(33): note: see reference to class template instantiation 'absl::lts_20250127::container_internal::raw_hash_set<Policy,Hash,Eq,Alloc>' being compiled
        with
        [
            Policy=absl::lts_20250127::container_internal::FlatHashMapPolicy<const google::protobuf::OneofDescriptor *,google::protobuf::compiler::java::OneofGeneratorInfo>,
            Hash=absl::lts_20250127::container_internal::HashEq<const google::protobuf::OneofDescriptor *,void>::Hash,
            Eq=absl::lts_20250127::container_internal::HashEq<const google::protobuf::OneofDescriptor *,void>::Eq,
            Alloc=std::allocator<std::pair<const google::protobuf::OneofDescriptor *const ,google::protobuf::compiler::java::OneofGeneratorInfo>>
        ]
external/abseil-cpp~\absl/container/internal/raw_hash_set.h(3030): note: see reference to class template instantiation 'absl::lts_20250127::conjunction<std::integral_constant<bool,true>,absl::lts_20250127::type_traits_internal::IsOwner<std::pair<const google::protobuf::OneofDescriptor *,google::protobuf::compiler::java::OneofGeneratorInfo>>,absl::lts_20250127::type_traits_internal::IsView<std::pair<const google::protobuf::OneofDescriptor *,google::protobuf::compiler::java::OneofGeneratorInfo>>>' being compiled
external/abseil-cpp~\absl/meta/type_traits.h(147): note: see reference to class template instantiation 'absl::lts_20250127::conjunction<absl::lts_20250127::type_traits_internal::IsOwner<std::pair<const google::protobuf::OneofDescriptor *,google::protobuf::compiler::java::OneofGeneratorInfo>>,absl::lts_20250127::type_traits_internal::IsView<std::pair<const google::protobuf::OneofDescriptor *,google::protobuf::compiler::java::OneofGeneratorInfo>>>' being compiled
external/abseil-cpp~\absl/meta/type_traits.h(147): note: see reference to class template instantiation 'absl::lts_20250127::type_traits_internal::IsOwner<std::pair<const google::protobuf::OneofDescriptor *,google::protobuf::compiler::java::OneofGeneratorInfo>>' being compiled
external/abseil-cpp~\absl/meta/type_traits.h(607): note: see reference to class template instantiation 'std::pair<const google::protobuf::OneofDescriptor *,google::protobuf::compiler::java::OneofGeneratorInfo>' being compiled
Target //:addressbook_cc_proto failed to build
INFO: Elapsed time: 7.979s, Critical Path: 7.80s
INFO: 33 processes: 33 internal.
ERROR: Build did NOT complete successfully

Anything else we should know about your project / environment

Here is the relevant error message.

error C2079: 'std::pair<const google::protobuf::FieldDescriptor *,google::protobuf::compiler::java::FieldGeneratorInfo>::second' uses undefined struct 'google::protobuf::compiler::java::FieldGeneratorInfo'

Here are the lines in question.

protobuf/src/google/protobuf/compiler/java/context.h

Lines 39 to 40 in 40ec76e

    
           struct FieldGeneratorInfo; 
        
           struct OneofGeneratorInfo;

protobuf/src/google/protobuf/compiler/java/context.h

Lines 80 to 83 in 40ec76e

    
           absl::flat_hash_map<const FieldDescriptor*, FieldGeneratorInfo> 
        
               field_generator_info_map_; 
        
           absl::flat_hash_map<const OneofDescriptor*, OneofGeneratorInfo> 
        
               oneof_generator_info_map_;

src/google/protobuf/compiler/java/context.h looks to be trying to pass a forward declared type to the second type parameter of absl::flat_hash_map, which I'm not quite sure if it's really supported or not.

Other observations

The compile failure is not reproducible with abseil-cpp 20240722.1. Looks like the issue is triggered by some recent change in abseil.
The compile failure is not reproducible with abseil-cpp 20250127.0 if you do not specify /Zc:__cplusplus compiler option.

The text was updated successfully, but these errors were encountered:

shaod2 · 2025-02-12T20:28:06Z

@sbenzaquen I think using forward declaration should be fine right? Is there something we can do here?

sbenzaquen · 2025-02-13T17:30:01Z

No, we should not use incomplete types in the container.
We should #include the right header instead of a forward declaration.
Or change the container declaration to do something else that does not require putting incomplete types directly on it.

Due to the build failure on Windows that happens when using protobuf with abseil 20250127.0 [1], we cannot yet update abseil. This commit updates the bzlmod depencencies as follows. abseil: 20240722.0.bcr.2 re2: 2024-07-02.bcr.1 googletest: 1.15.2 [1]: protocolbuffers/protobuf#20331

Due to the build failure on Windows that happens when using protobuf with abseil 20250127.0 [1], we cannot yet update abseil. This commit updates the bzlmod depencencies as follows. abseil: 20240722.0.bcr.2 re2: 2024-07-02.bcr.1 protobuf: 29.3 googletest: 1.15.2 [1]: protocolbuffers/protobuf#20331

* To fix build failures on Windows with Bazel. + protocolbuffers/protobuf#20331 * abseil-cpp: 20240722.0.bcr.2 * re2: 2024-07-02.bcr.1 * protobuf: 29.3 * googletest: 1.15.2 PiperOrigin-RevId: 729343696

With this commit we start using 'clang-cl' to build Mozc for Windows with Bazel. While the original motivation of switching to 'clang-cl' is that protobuf is planning to stop supporting cl.exe with bazel build in protobuf v34 [1], but there is also an on-going build breakage that happens when combining protobuf and abseil-cpp 20250127.0, which is not yet fixed [2]. The latter one is more critical for us because more and more bzlmod dependencies start requiring abseil-cpp 20250127.0 or higher. There must be no impact on Windows GYP build. Closes google#1179. [1]: protocolbuffers/protobuf#20085 [2]: protocolbuffers/protobuf#20331

yukawa · 2025-02-27T05:19:59Z

Updating the title to make it clear that this is now obvious in v30.0-rc2, which starts requiring abseil-cpp 20250127.0.

With this commit we start using 'clang-cl' to build Mozc for Windows with Bazel. While the original motivation of switching to 'clang-cl' is that protobuf is planning to stop supporting cl.exe with bazel build in protobuf v34 [1], there is also an on-going build breakage that happens when combining protobuf and abseil-cpp 20250127.0, which is not yet fixed [2]. The latter one is more critical for us because more and more bzlmod dependencies start requiring abseil-cpp 20250127.0 or higher. There must be no impact on Windows GYP build. Closes google#1179. [1]: protocolbuffers/protobuf#20085 [2]: protocolbuffers/protobuf#20331

With this commit we start using 'clang-cl' to build Mozc for Windows with Bazel. While the original motivation of switching to 'clang-cl' is that protobuf is planning to stop supporting cl.exe with bazel build in protobuf v34 [1], there is also an on-going build breakage that happens when combining protobuf and abseil-cpp 20250127.0, which is not yet fixed [2]. The latter one is more critical for us because more and more bzlmod dependencies start requiring abseil-cpp 20250127.0 or higher. There must be no impact on Windows GYP build. Closes #1179. [1]: protocolbuffers/protobuf#20085 [2]: protocolbuffers/protobuf#20331 PiperOrigin-RevId: 736417401

As discussed in google#1208, protobuf 30.0 does not compile on macOS, which impiles that clang 16 shipped with Xcode 16.2 is also affected by the following issue. * protocolbuffers/protobuf#20645 Let's update comments in 'MODULE.bazel' to refer to the above issue before closing google#1177. Note that the following open issue is no longer the case as we have finally migrated to 'clang-cl' (google#1179). * protocolbuffers/protobuf#20331

As discussed in google#1208, protobuf 30.0 does not compile on macOS, which implies that clang 16 shipped with Xcode 16.2 is also affected by the following issue. * protocolbuffers/protobuf#20645 Let's update comments in 'MODULE.bazel' to refer to the above issue before closing google#1177. Note that the following open issue is no longer the case as we have finally migrated to 'clang-cl' (google#1179). * protocolbuffers/protobuf#20331

Now that Windows Bazel build uses 'clang-cl' (google#1179), we should be able to switch to update 'abseil-cpp' to 20250127.0 without worrying about the following issue. * protocolbuffers/protobuf#20331 Note that GYP build remain unchanged until the corresponding abseil git submodule is updated. No observable behavior change is intended in the final artifacts. Closes google#1172.

As discussed in google#1208, protobuf 30.0 does not compile on macOS, which implies that clang 16 shipped with Xcode 16.2 is also affected by the following issue. * protocolbuffers/protobuf#20645 Let's update comments in 'MODULE.bazel' to refer to the above issue before closing google#1177. Note that the following open issue is no longer the case as we have finally migrated to 'clang-cl' (google#1179). * protocolbuffers/protobuf#20331

As discussed in #1208, protobuf 30.0 does not compile on macOS, which implies that clang 16 shipped with Xcode 16.2 is also affected by the following issue. * protocolbuffers/protobuf#20645 Let's update comments in 'MODULE.bazel' to refer to the above issue before closing #1177. Note that the following open issue is no longer the case as we have finally migrated to 'clang-cl' (#1179). * protocolbuffers/protobuf#20331 PiperOrigin-RevId: 736914763

There is a KI, protocolbuffers/protobuf#20331, when using protobuf + newer versions of abseil that causes builds to fail on MSVC. This change uses full structs instead of forward declarations to work around this. I expect that this patch can be removed in the future when protobuf fixes this issue. Bug: 340578705 Change-Id: I54c3938ba098845587d7f7206e32c87af61283c3 Reviewed-on: https://chromium-review.googlesource.com/c/chromium/src/+/6427203 Auto-Submit: Ryan Harrison <[email protected]> Reviewed-by: Steven Holte <[email protected]> Commit-Queue: Steven Holte <[email protected]> Cr-Commit-Position: refs/heads/main@{#1441759}

yukawa added the untriaged auto added to all issues by default when created. label Feb 12, 2025

yukawa mentioned this issue Feb 12, 2025

Update third_party/abseil-cpp to 20250127.0 or later google/mozc#1172

Closed

shaod2 assigned sbenzaquen Feb 12, 2025

shaod2 added c++ and removed untriaged auto added to all issues by default when created. labels Feb 12, 2025

yukawa mentioned this issue Feb 26, 2025

Start using clang-cl for Windows Bazel build google/mozc#1198

Merged

yukawa mentioned this issue Mar 13, 2025

Explain why protobuf 30.0 cannot be used google/mozc#1210

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[v30.0-rc2 regression] `src/google/protobuf/compiler/java/context.h` causes compile error C2079 in MSVC with `/std:c++20` and `/Zc:__cplusplus` #20331

[v30.0-rc2 regression] `src/google/protobuf/compiler/java/context.h` causes compile error C2079 in MSVC with `/std:c++20` and `/Zc:__cplusplus` #20331

yukawa commented Feb 12, 2025

shaod2 commented Feb 12, 2025

Uh oh!

sbenzaquen commented Feb 13, 2025

Uh oh!

yukawa commented Feb 27, 2025

Uh oh!

[v30.0-rc2 regression] src/google/protobuf/compiler/java/context.h causes compile error C2079 in MSVC with /std:c++20 and /Zc:__cplusplus #20331

[v30.0-rc2 regression] src/google/protobuf/compiler/java/context.h causes compile error C2079 in MSVC with /std:c++20 and /Zc:__cplusplus #20331

Comments

yukawa commented Feb 12, 2025

Other observations

shaod2 commented Feb 12, 2025

Uh oh!

sbenzaquen commented Feb 13, 2025

Uh oh!

yukawa commented Feb 27, 2025

Uh oh!

[v30.0-rc2 regression] `src/google/protobuf/compiler/java/context.h` causes compile error C2079 in MSVC with `/std:c++20` and `/Zc:__cplusplus` #20331

[v30.0-rc2 regression] `src/google/protobuf/compiler/java/context.h` causes compile error C2079 in MSVC with `/std:c++20` and `/Zc:__cplusplus` #20331