[MSVC] Error "relocation against symbol in discarded section: $LN" linking against prebuilt Firebase C++ SDK

[triplef]

Linking an executable built with Clang against the prebuilt Firebase C++ SDK static libraries built using Visual Studio 2022 results in linker errors like:

lld-link: error: relocation against symbol in discarded section: $LN10

This was reproduced with the official Clang 16.0.1 release as well as older Clang releases, and happens both when using clang.exe or the Clang-CL driver. Using the Visual Studio compiler and linker works fine.

Here is a minimal sample project using CMake and v10.7.0 of the Firebase SDK:
firebase-cpp-sdk-test.zip

This was previously also reported as firebase/firebase-cpp-sdk#793, but I believe this is likely a Clang issue.

Full error output

$ cmd.exe /C "cd . && C:\PROGRA~1\LLVM\bin\CLANG_~1.EXE -fuse-ld=lld-link -nostartfiles -nostdlib -g -Xclang -gcodeview -O0 -D_DEBUG -D_DLL -D_MT -Xclang --dependent-lib=msvcrtd -Xlinker /subsystem:console CMakeFiles/test.dir/test.cpp.obj -o test.exe -Xlinker /MANIFEST:EMBED -Xlinker /implib:test.lib -Xlinker /pdb:test.pdb -Xlinker /version:0.0   C:/Users/frede/Dev/firebase-cpp-sdk-test/firebase_cpp_sdk/libs/windows/VS2019/MD/x64/Debug/firebase_app.lib  -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32 -lole32 -loleaut32 -luuid -lcomdlg32 -ladvapi32 -loldnames  && cd ."
lld-link: error: relocation against symbol in discarded section: $LN10
>>> referenced by C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.34.31933\include\vector:917
>>>               firebase_app.lib(d94e1bff3c0e8414a6679ea91c3e103e_firebase_app.dir_Debug_util.obj):(__catch$??$_Emplace_reallocate@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@?$vector@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@V?$allocator@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@2@@std@@AEAAPEAV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@1@QEAV21@$$QEAV21@@Z$0)

lld-link: error: relocation against symbol in discarded section: $LN10
>>> referenced by C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.34.31933\include\vector:917
>>>               firebase_app.lib(d94e1bff3c0e8414a6679ea91c3e103e_firebase_app.dir_Debug_variant_util.obj):(__catch$??$_Emplace_reallocate@AEBQEAUNamespace@f_b_flatbuffers@@@?$vector@PEAUNamespace@f_b_flatbuffers@@V?$allocator@PEAUNamespace@f_b_flatbuffers@@@std@@@std@@AEAAPEAPEAUNamespace@f_b_flatbuffers@@QEAPEAU23@AEBQEAU23@@Z$0)

lld-link: error: relocation against symbol in discarded section: $LN36
>>> referenced by C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.34.31933\include\vector:1279
>>>               firebase_app.lib(d94e1bff3c0e8414a6679ea91c3e103e_firebase_app.dir_Debug_variant_util.obj):($LN34)

lld-link: error: relocation against symbol in discarded section: $LN10
>>> referenced by C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.34.31933\include\vector:917
>>>               firebase_app.lib(d94e1bff3c0e8414a6679ea91c3e103e_firebase_app.dir_Debug_variant_util.obj):(__catch$??$_Emplace_reallocate@AEBUValue@Builder@f_b_flexbuffers@@@?$vector@UValue@Builder@f_b_flexbuffers@@V?$allocator@UValue@Builder@f_b_flexbuffers@@@std@@@std@@AEAAPEAUValue@Builder@f_b_flexbuffers@@QEAU234@AEBU234@@Z$0)

lld-link: error: relocation against symbol in discarded section: $LN10
>>> referenced by C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.34.31933\include\vector:917
>>>               firebase_app.lib(d94e1bff3c0e8414a6679ea91c3e103e_firebase_app.dir_Debug_heartbeat_storage_desktop.obj):(__catch$??$_Emplace_reallocate@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@?$vector@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@V?$allocator@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@2@@std@@AEAAPEAV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@1@QEAV21@$$QEAV21@@Z$0)

lld-link: error: relocation against symbol in discarded section: $LN8
>>> referenced by C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.34.31933\include\vector:1608
>>>               firebase_app.lib(d94e1bff3c0e8414a6679ea91c3e103e_firebase_app.dir_Debug_variant_util.obj):(__catch$??$_Resize_reallocate@U_Value_init_tag@std@@@?$vector@EV?$allocator@E@std@@@std@@AEAAX_KAEBU_Value_init_tag@1@@Z$0)

lld-link: error: relocation against symbol in discarded section: $LN10
>>> referenced by C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.34.31933\include\vector:917
>>>               firebase_app.lib(d94e1bff3c0e8414a6679ea91c3e103e_firebase_app.dir_Debug_variant_util.obj):(__catch$??$_Emplace_reallocate@E@?$vector@EV?$allocator@E@std@@@std@@AEAAPEAEQEAE$$QEAE@Z$0)

lld-link: error: relocation against symbol in discarded section: $LN10
>>> referenced by C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.34.31933\include\vector:917
>>>               firebase_app.lib(d94e1bff3c0e8414a6679ea91c3e103e_firebase_app.dir_Debug_variant_util.obj):(__catch$??$_Emplace_reallocate@UValue@Builder@f_b_flexbuffers@@@?$vector@UValue@Builder@f_b_flexbuffers@@V?$allocator@UValue@Builder@f_b_flexbuffers@@@std@@@std@@AEAAPEAUValue@Builder@f_b_flexbuffers@@QEAU234@$$QEAU234@@Z$0)

lld-link: error: relocation against symbol in discarded section: $LN8
>>> referenced by C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.34.31933\include\vector:1608
>>>               firebase_app.lib(d94e1bff3c0e8414a6679ea91c3e103e_firebase_app.dir_Debug_variant_util.obj):(__catch$??$_Resize_reallocate@U_Value_init_tag@std@@@?$vector@UValue@Builder@f_b_flexbuffers@@V?$allocator@UValue@Builder@f_b_flexbuffers@@@std@@@std@@AEAAX_KAEBU_Value_init_tag@1@@Z$0)

lld-link: error: relocation against symbol in discarded section: $LN10
>>> referenced by C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.34.31933\include\vector:917
>>>               firebase_app.lib(d94e1bff3c0e8414a6679ea91c3e103e_firebase_app.dir_Debug_variant_util.obj):(__catch$??$_Emplace_reallocate@VVariant@firebase@@@?$vector@VVariant@firebase@@V?$allocator@VVariant@firebase@@@std@@@std@@AEAAPEAVVariant@firebase@@QEAV23@$$QEAV23@@Z$0)

lld-link: error: relocation against symbol in discarded section: $LN28
>>> referenced by C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.34.31933\include\vector:1221
>>>               firebase_app.lib(d94e1bff3c0e8414a6679ea91c3e103e_firebase_app.dir_Debug_variant_util.obj):(__catch$??$_Insert_counted_range@PEBE@?$vector@EV?$allocator@E@std@@@std@@AEAAXV?$_Vector_const_iterator@V?$_Vector_val@U?$_Simple_types@E@std@@@std@@@1@PEBE_K@Z$0)

lld-link: error: relocation against symbol in discarded section: $LN10
>>> referenced by C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.34.31933\include\vector:917
>>>               firebase_app.lib(d3d7d08a438878e74aeb2cbaaedfc967_flatbuffers.dir_Debug_idl_parser.obj):(__catch$??$_Emplace_reallocate@AEBQEAUNamespace@f_b_flatbuffers@@@?$vector@PEAUNamespace@f_b_flatbuffers@@V?$allocator@PEAUNamespace@f_b_flatbuffers@@@std@@@std@@AEAAPEAPEAUNamespace@f_b_flatbuffers@@QEAPEAU23@AEBQEAU23@@Z$0)

lld-link: error: relocation against symbol in discarded section: $LN10
>>> referenced by C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.34.31933\include\vector:917
>>>               firebase_app.lib(d3d7d08a438878e74aeb2cbaaedfc967_flatbuffers.dir_Debug_idl_parser.obj):(__catch$??$_Emplace_reallocate@AEBV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@?$vector@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@V?$allocator@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@@2@@std@@AEAAPEAV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@1@QEAV21@AEBV21@@Z$0)

lld-link: error: relocation against symbol in discarded section: $LN32
>>> referenced by C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.34.31933\include\vector:1255
>>>               firebase_app.lib(d94e1bff3c0e8414a6679ea91c3e103e_firebase_app.dir_Debug_variant_util.obj):($LN30)

lld-link: error: relocation against symbol in discarded section: $LN29
>>> referenced by C:\Program Files\Microsoft Visual Studio\2022\Enterprise\VC\Tools\MSVC\14.34.31933\include\vector:1125
>>>               firebase_app.lib(d94e1bff3c0e8414a6679ea91c3e103e_firebase_app.dir_Debug_variant_util.obj):(__catch$?insert@?$vector@EV?$allocator@E@std@@@std@@QEAA?AV?$_Vector_iterator@V?$_Vector_val@U?$_Simple_types@E@std@@@std@@@2@V?$_Vector_const_iterator@V?$_Vector_val@U?$_Simple_types@E@std@@@std@@@2@_KAEBE@Z$0)

[llvmbot]

@llvm/issue-subscribers-lld-coff

[triplef]

@llvm/issue-subscribers-lld-coff are there any linker flags I could try to work around this issue, or anything else that would help to debug this further? Thanks! 🙏

[rnk]

+@zmodem @amykhuang

[compnerd]

I'm fairly certain that the symbols being referenced are debug symbol references. It seems that -debug:dwarf does not ignore the CodeView that is present in the object files, and that results in the undefined references.

[triplef]

Thanks @compnerd, so if I understand correctly you suspect this is to be a bug in lld-link?

Any idea why this might be happening with these libraries specifically, and is there anything Firebase could change in the build process to prevent this?

[compnerd]

Thanks @compnerd, so if I understand correctly you suspect this is to be a bug in lld-link?

Any idea why this might be happening with these libraries specifically, and is there anything Firebase could change in the build process to prevent this?

The RelWithDebInfo builds emit codeview so that debugging is possible. A pure release build would not. However, we should understand why lld is ending up preserving the line information with -debug:dwarf as well.

[triplef]

I’ve build a bunch of libraries with Visual Studio using RelWithDebInfo and linked them with lld, and never came across this issue with any other libraries, so it’s still unclear to me what’s special about the Firebase libraries.

Do you know if there’s a workaround we could use, e.g. is there a way to strip codeview before linking?

[compnerd]

I’ve build a bunch of libraries with Visual Studio using RelWithDebInfo and linked them with lld, and never came across this issue with any other libraries, so it’s still unclear to me what’s special about the Firebase libraries.

oh? That’s good to know! I suspect something else is keeping the symbol alive then. That’s good to know.

Do you know if there’s a workaround we could use, e.g. is there a way to strip codeview before linking?

I’ve been unable to find one, other than rebuilding. There’s no strip utility that’s provided with MSVC, and llvm-strip is meant to work on final binaries though it should work on object files. I’ve had issues with stripping on windows so I tend to avoid using it there.

[mstorsjo]

I’ve build a bunch of libraries with Visual Studio using RelWithDebInfo and linked them with lld, and never came across this issue with any other libraries, so it’s still unclear to me what’s special about the Firebase libraries.

oh? That’s good to know! I suspect something else is keeping the symbol alive then. That’s good to know.

Do you know if there’s a workaround we could use, e.g. is there a way to strip codeview before linking?

I’ve been unable to find one, other than rebuilding. There’s no strip utility that’s provided with MSVC, and llvm-strip is meant to work on final binaries though it should work on object files.

llvm-strip does work on both object files and executables. The default actions might be a bit too harsh on object files, but I believe the -S option should remove any section that starts with .debug, which iirc matches both DWARF and codeview debug info.

[triplef]

I’m getting this error when trying to strip these libraries – any ideas?

$ llvm-strip -S firebase_app.lib
llvm-strip.exe: error: 'firebase_app.lib': unexpected associative section index

[compnerd]

Intriguing! It seems like a __declspec(select...) gone awry? This might be a MSVC bug? The release itself could've been built with a compiler that emitted a slightly questionable object file that link happily consumed but lld does not?

[triplef]

According to Firebase project’s GitHub Actions workflow they are building with Visual Studio 2022. They do have a somewhat involved build process with multiple internal dependencies, so something in there must be causing this.

However when I build the Firebase libraries myself using Visual Studio 2022 the resulting libraries link just fine. Unfortunately I cannot say what exactly I’m doing differently, or whether it might be caused by a different build environment.

Regarding the error from llvm-strip there’s issue #53433 (comment) mentioning this:

It appears that this error is triggered when PointerToSymbolTable is set in the PE IMAGE_FILE_HEADER. If this is zeroed out, then llvm-strip will work fine. Hope this helps...

[compnerd]

The PointerToSymbolTable should not be zero - it is an object file, not a linked module.

One thing that stands out is that the referenced COMDATs are all STL related. It might be a mismatch in the version of msvcprt that it was built against and the version being used for the application.

[rnk]

This looks WinEH related, I'm not seeing a clear indication that this is codeview related. The sections containing the reference are the catch handler sections (__catch$*), which will be in .text$x. The $LN symbols are probably references back to the parent function .text section, and the linker decided that the parent section should be dropped for one reason or another.

I would try disabling ICF and section GC, so /OPT:NOREF,NOICF or something similar, and see if the issue goes away. If so, there must be some bug in our handling of associated comdat sections there.

[compnerd]

Nice catch @rnk! I didn't notice the __catch at all.

[compnerd]

Just verified that -opt:noref,noicf has no impact and can still reproduce the relocation against the discarded section.

[aganea]

I used to see this problem a long while ago at Ubisoft, but never got to the end of it. And then it went away. Thanks for the reproducer @triplef ! I managed to take a look at it again and I have a fix.

The issue is both a problem in LLD and in MSVC. In LLD, we don't support label symbols (that is the $LN symbols reported by the linker) in the same way that link.exe does. But then the root problem seems to be a bug in MSVC: in some unknown conditions, it manages to generate a COMDAT .text$mn (that is, a function block) and its associated .text$x section (that is, a catch funclet), but the latter is missing its COMDAT and "associative" selection flag. Clang doesn't generate the .text$x - but we discussed with @rnk to maybe doing this.

The following small repro is close to triggering the issue, but I can't figure out what makes MSVC to "forget" the COMDAT for .text$x:

C:\test>cat a.h
inline void func(int *a) {
  try {
    *a = 0;
  }
  catch (...) {
        throw;
  }
}

C:\test>cat a.cpp
#include "a.h"
void a() {
   func(nullptr);
}

C:\test>cat b.cpp
#include "a.h"
void b() {
   func(nullptr);
}

C:\test>cl /c /EHsc a.cpp
C:\test>cl /c /EHsc b.cpp

(now for both `a.obj` and `b.obj`, manually remove IMAGE_SCN_LNK_COMDAT from the `.text$x` section definition, and IMAGE_COMDAT_SELECT_ASSOCIATIVE from the section symbol)

C:\test>lld-link a.obj b.obj /entry:a /subsystem:console /force:unresolved
...
lld-link: error: relocation against symbol in discarded section: $LN7

I've posted https://reviews.llvm.org/D157136

[aganea]

Just for completeness, the issue is not with MSVC like I wrote previously, but with binutils objcopy which is used to package the firebase-sdk.

[mstorsjo]

but with binutils objdump

objcopy, not objdump. But that does indeed explain the weird object files!

[aganea]

Yes, objcopy! 😄 Fixed the comment.