Linking failed when building on Windows 10

[HolyWu]

Bug Description

At first the compilation failed with:

Starting local Bazel server and connecting to it...
ERROR: C:/users/holy/downloads/trtorch/cpp/lib/BUILD:34:10: no such package 'cpp/api': BUILD file not found in any of the following directories. Add a BUILD file to a directory to mark it as a package.
 - C:/users/holy/downloads/trtorch/cpp/api and referenced by '//cpp/lib:trtorch.dll'
ERROR: Analysis of target '//:libtrtorch' failed; build aborted: Analysis failed
INFO: Elapsed time: 9.991s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (31 packages loaded, 73 targets configured)
    Fetching @local_config_cc; Restarting.

I removed /api at https://github.com/NVIDIA/TRTorch/blob/f99a6ca763eb08982e8b7172eb948a090bcbf11c/cpp/lib/BUILD#L40 and that error was gone. But then the linking failed with:

ERROR: C:/users/holy/downloads/trtorch/cpp/lib/BUILD:34:10: Linking cpp/lib/trtorch.dll failed: (Exit 1120): link.exe failed: error executing command C:/Program Files (x86)/Microsoft Visual Studio/2019/Community/VC/Tools/MSVC/14.29.30133/bin/HostX64/x64/link.exe @bazel-out/x64_windows-opt/bin/cpp/lib/trtorch.dll-2.params
LINK : warning LNK4044: unrecognized option '/lpthread'; ignored
LINK : warning LNK4044: unrecognized option '/lpthread'; ignored
LINK : warning LNK4044: unrecognized option '/Wl,-rpath,lib/'; ignored
   Creating library bazel-out/x64_windows-opt/bin/cpp/lib/trtorch.dll.if.lib and object bazel-out/x64_windows-opt/bin/cpp/lib/trtorch.dll.if.exp
runtime.lo.lib(TRTEngine.obj) : error LNK2019: unresolved external symbol createInferRuntime_INTERNAL referenced in function "public: __cdecl trtorch::core::runtime::TRTEngine::TRTEngine(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >,class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> >,struct trtorch::core::runtime::CudaDevice)" (??0TRTEngine@runtime@core@trtorch@@QEAA@V?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@std@@0UCudaDevice@123@@Z)
converters.lo.lib(activation.obj) : error LNK2019: unresolved external symbol getPluginRegistry referenced in function "class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > __cdecl std::operator+<char,struct std::char_traits<char>,class std::allocator<char> >(class std::basic_string<char,struct std::char_traits<char>,class std::allocator<char> > &&,char const * const)" (??$?HDU?$char_traits@D@std@@V?$allocator@D@1@@std@@YA?AV?$basic_string@DU?$char_traits@D@std@@V?$allocator@D@2@@0@$$QEAV10@QEBD@Z)
trtorch_plugins.lo.lib(register_plugins.obj) : error LNK2001: unresolved external symbol getPluginRegistry
trtorch_plugins.lo.lib(normalize_plugin.obj) : error LNK2001: unresolved external symbol getPluginRegistry
trtorch_plugins.lo.lib(interpolate_plugin.obj) : error LNK2001: unresolved external symbol getPluginRegistry
converters.lo.lib(pooling.obj) : error LNK2001: unresolved external symbol getPluginRegistry
converters.lo.lib(normalize.obj) : error LNK2001: unresolved external symbol getPluginRegistry
converters.lo.lib(interpolate.obj) : error LNK2001: unresolved external symbol getPluginRegistry
converters.lo.lib(batch_norm.obj) : error LNK2001: unresolved external symbol getPluginRegistry
trtorch_plugins.lo.lib(register_plugins.obj) : error LNK2019: unresolved external symbol initLibNvInferPlugins referenced in function "public: __cdecl trtorch::core::plugins::impl::TRTorchPluginRegistry::TRTorchPluginRegistry(void)" (??0TRTorchPluginRegistry@impl@plugins@core@trtorch@@QEAA@XZ)
conversionctx.lib(ConversionCtx.obj) : error LNK2019: unresolved external symbol createInferBuilder_INTERNAL referenced in function "public: __cdecl trtorch::core::conversion::ConversionCtx::ConversionCtx(struct trtorch::core::conversion::BuilderSettings)" (??0ConversionCtx@conversion@core@trtorch@@QEAA@UBuilderSettings@123@@Z)
bazel-out\x64_windows-opt\bin\cpp\lib\trtorch.dll : fatal error LNK1120: 4 unresolved externals
Target //:libtrtorch failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 180.866s, Critical Path: 29.70s
INFO: 729 processes: 605 internal, 124 local.
FAILED: Build did NOT complete successfully

The missing symbols createInferRuntime_INTERNAL, getPluginRegistry, initLibNvInferPlugins and createInferBuilder_INTERNAL are all in TensorRT. But looking at bazel-out/x64_windows-opt/bin/cpp/lib/trtorch.dll-2.params, it didn't link with any library from TensorRT.

/nologo
/DLL
/OUT:bazel-out/x64_windows-opt/bin/cpp/lib/trtorch.dll
/IMPLIB:bazel-out/x64_windows-opt/bin/cpp/lib/trtorch.dll.if.lib
/WHOLEARCHIVE:bazel-out/x64_windows-opt/bin/cpp/trtorch.lo.lib
/WHOLEARCHIVE:bazel-out/x64_windows-opt/bin/core/core.lo.lib
/WHOLEARCHIVE:bazel-out/x64_windows-opt/bin/core/runtime/runtime.lo.lib
/WHOLEARCHIVE:bazel-out/x64_windows-opt/bin/core/partitioning/partitioning.lo.lib
bazel-out/x64_windows-opt/bin/core/conversion/conversion.lib
/WHOLEARCHIVE:bazel-out/x64_windows-opt/bin/core/conversion/converters/converters.lo.lib
/WHOLEARCHIVE:bazel-out/x64_windows-opt/bin/core/plugins/trtorch_plugins.lo.lib
/WHOLEARCHIVE:bazel-out/x64_windows-opt/bin/core/conversion/evaluators/evaluators.lo.lib
/WHOLEARCHIVE:bazel-out/x64_windows-opt/bin/core/conversion/var/var.lo.lib
/WHOLEARCHIVE:bazel-out/x64_windows-opt/bin/core/conversion/converters/converter_util.lo.lib
/WHOLEARCHIVE:bazel-out/x64_windows-opt/bin/core/conversion/converters/weights.lo.lib
bazel-out/x64_windows-opt/bin/core/conversion/conversionctx/conversionctx.lib
/WHOLEARCHIVE:bazel-out/x64_windows-opt/bin/core/conversion/tensorcontainer/tensorcontainer.lo.lib
/WHOLEARCHIVE:bazel-out/x64_windows-opt/bin/core/lowering/lowering.lo.lib
bazel-out/x64_windows-opt/bin/core/lowering/passes/passes.lib
bazel-out/x64_windows-opt/bin/core/ir/ir.lib
bazel-out/x64_windows-opt/bin/core/util/trt_util.lib
bazel-out/x64_windows-opt/bin/core/util/exception.lib
bazel-out/x64_windows-opt/bin/core/util/logging/logging.lib
external/cuda/lib/x64/cudart.lib
external/cuda/lib/x64/cublas.lib
external/libtorch/lib/torch.lib
external/libtorch/lib/torch_cuda.lib
external/libtorch/lib/torch_cpu.lib
external/libtorch/lib/c10_cuda.lib
external/libtorch/lib/c10.lib
/SUBSYSTEM:CONSOLE
-lpthread
-lpthread
-Wl,-rpath,lib/
/MACHINE:X64
/DEFAULTLIB:msvcrt.lib
/OPT:ICF
/OPT:REF
/DEF:bazel-out/x64_windows-opt/bin/cpp/lib/trtorch.dll.gen.empty.def
/ignore:4070

So, what's the culprit?

Environment

• TRTorch Version: git latest master branch f99a6ca
• PyTorch Version: 1.9.1
• CPU Architecture: x86_64
• OS: Windows 10
• How you installed PyTorch: pip
• Build command you used: bazel build //:libtrtorch -c opt
• Are you using local sources or building from archives: local sources
• Python version: 3.9.7
• CUDA version: 11.1

Additional context

The content of my WORKSPACE:

workspace(name = "TRTorch")

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
load("@bazel_tools//tools/build_defs/repo:git.bzl", "git_repository")

http_archive(
    name = "rules_python",
    sha256 = "778197e26c5fbeb07ac2a2c5ae405b30f6cb7ad1f5510ea6fdac03bded96cc6f",
    url = "https://github.com/bazelbuild/rules_python/releases/download/0.2.0/rules_python-0.2.0.tar.gz",
)

load("@rules_python//python:pip.bzl", "pip_install")

http_archive(
    name = "rules_pkg",
    sha256 = "038f1caa773a7e35b3663865ffb003169c6a71dc995e39bf4815792f385d837d",
    urls = [
        "https://mirror.bazel.build/github.com/bazelbuild/rules_pkg/releases/download/0.4.0/rules_pkg-0.4.0.tar.gz",
        "https://github.com/bazelbuild/rules_pkg/releases/download/0.4.0/rules_pkg-0.4.0.tar.gz",
    ],
)

load("@rules_pkg//:deps.bzl", "rules_pkg_dependencies")

rules_pkg_dependencies()

git_repository(
    name = "googletest",
    commit = "703bd9caab50b139428cea1aaff9974ebee5742e",
    remote = "https://github.com/google/googletest",
    shallow_since = "1570114335 -0400",
)

# External dependency for trtorch if you already have precompiled binaries.
# This is currently used in pytorch NGC container CI testing.
local_repository(
    name = "trtorch",
    path = "C:/Python39/Lib/site-packages/trtorch"
)

# CUDA should be installed on the system locally
new_local_repository(
    name = "cuda",
    build_file = "@//third_party/cuda:BUILD",
    path = "C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.1",
)

new_local_repository(
    name = "cublas",
    build_file = "@//third_party/cublas:BUILD",
    path = "C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.1",
)

####################################################################################
# Locally installed dependencies (use in cases of custom dependencies or aarch64)
####################################################################################

# NOTE: In the case you are using just the pre-cxx11-abi path or just the cxx11 abi path
# with your local libtorch, just point deps at the same path to satisfy bazel.

# NOTE: NVIDIA's aarch64 PyTorch (python) wheel file uses the CXX11 ABI unlike PyTorch's standard
# x86_64 python distribution. If using NVIDIA's version just point to the root of the package
# for both versions here and do not use --config=pre-cxx11-abi

new_local_repository(
    name = "libtorch",
    path = "C:/Python39/Lib/site-packages/torch",
    build_file = "third_party/libtorch/BUILD"
)

new_local_repository(
    name = "libtorch_pre_cxx11_abi",
    path = "C:/Python39/Lib/site-packages/torch",
    build_file = "third_party/libtorch/BUILD"
)

new_local_repository(
    name = "cudnn",
    path = "C:/Program Files/NVIDIA GPU Computing Toolkit/CUDA/v11.1",
    build_file = "@//third_party/cudnn/local:BUILD"
)

new_local_repository(
   name = "tensorrt",
   path = "C:/TensorRT-8.0.3.4",
   build_file = "@//third_party/tensorrt/local:BUILD"
)

[wasif-manto]

I have the same issue . Any update?

[JiangZongKang]

I have the same issue . Any update?

Have you solved it ?

[wasif-manto]

Nope :(

[yuriishutkin]

On Windows even if some module (here tensorrt) is used as shared library, dll reference .lib still had to be linked into target application.

This patch helps to solve unresolved externals:

diff --git a/third_party/tensorrt/local/BUILD b/third_party/tensorrt/local/BUILD
index 7b4fcc28..79ddbacb 100644
--- a/third_party/tensorrt/local/BUILD
+++ b/third_party/tensorrt/local/BUILD
@@ -89,7 +89,7 @@ cc_library(
         "@cuda//:cudart",
         "@cudnn",
     ] + select({
-        ":windows": ["@cuda//:cublas"],
+        ":windows": ["@cuda//:cublas", "nvinfer_static_lib"],
         "//conditions:default": ["@cuda//:cublas"],
     }),
 )
@@ -295,7 +295,7 @@ cc_library(
     }),
     srcs = select({
         ":aarch64_linux": ["lib/aarch64-linux-gnu/libnvinfer_plugin.so"],
-        ":windows": ["lib/nvinfer_plugin.dll"],
+        ":windows": ["lib/nvinfer_plugin.lib", "lib/nvinfer_plugin.dll"],
         "//conditions:default": ["lib/x86_64-linux-gnu/libnvinfer_plugin.so"],
     }),
     includes = select({

[github-actions]

This issue has not seen activity for 90 days, Remove stale label or comment or this will be closed in 10 days

[ncomly-nvidia]

Please try #1058 for Windows Support