Skip to content

build.sh not able to run due to python dependency issues #723

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
BenjaminBraunDev opened this issue Mar 11, 2025 · 0 comments
Open

build.sh not able to run due to python dependency issues #723

BenjaminBraunDev opened this issue Mar 11, 2025 · 0 comments

Comments

@BenjaminBraunDev
Copy link

BenjaminBraunDev commented Mar 11, 2025 •
edited
Loading

When building from source following the instructions in build.md I am running into issue with python build dependences, certain packages are not recognized.

Specifically these libraries have errors:

torch-2.6.0a0+ecf3bae40a.nv25.1.dist-info
pytorch_triton-3.1.0+cf34004b8.internal.dist-info
flash_attn-2.4.2.dev3.dist-info
torch-2.6.0a0+ecf3bae40a.nv25.1.dist-info
pytorch_triton-3.1.0+cf34004b8.internal.dist-info
flash_attn-2.4.2.dev3.dist-info

I've tried both the unmodified build.sh script, and replacing the image versions for container image 25.02, though both ran into the same errors. In fact, I got even more python version errors when setting changing the image versions to 25.02.

Errors with container images for 25.02:

torch-2.6.0a0+ecf3bae40a.nv25.1.dist-info
torchvision-0.20.0a0.dist-info
pytorch_triton-3.1.0+cf34004b8.internal.dist-info
jinja2-3.1.4.dist-info
sympy-1.13.1.dist-info
flash_attn-2.4.2.dev3.dist-info
torch-2.6.0a0+ecf3bae40a.nv25.1.dist-info
torchvision-0.20.0a0.dist-info
pytorch_triton-3.1.0+cf34004b8.internal.dist-info
jinja2-3.1.4.dist-info
sympy-1.13.1.dist-info
flash_attn-2.4.2.dev3.dist-info

Setup

  • Ubuntu 24.04.2 LTS
  • Dependencies installed with versions corresponding to the 25.02 container image in the dependency matrix (CUDA 12.8, etc.)
  • I've built the tritonserver image from the 25.02-trtllm-python-py3 base image on this same machine using the build.py script in the server repo without issues

Full Error Log

(my-env) benjaminbraun_google_com@triton-orca-refactor:~/tensorrtllm_backend$ ./build.sh 
Updated Git hooks.
Git LFS initialized.
2025/03/11 21:11:59 in: []string{}
2025/03/11 21:11:59 Parsed entitlements: []
[+] Building 1.2s (52/78)                                                                                                                                                                docker:default
 => [internal] load build definition from Dockerfile.triton.trt_llm_backend                                                                                                                        0.0s
 => => transferring dockerfile: 12.62kB                                                                                                                                                            0.0s
 => WARN: FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 8)                                                                                                                     0.0s
 => WARN: FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 9)                                                                                                                     0.0s
 => WARN: FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 106)                                                                                                                   0.0s
 => WARN: FromAsCasing: 'as' and 'FROM' keywords' casing do not match (line 122)                                                                                                                   0.0s
 => [internal] load metadata for nvcr.io/nvidia/tritonserver:24.11-py3-min                                                                                                                         0.2s
 => [internal] load metadata for nvcr.io/nvidia/pytorch:24.11-py3                                                                                                                                  0.2s
 => [internal] load .dockerignore                                                                                                                                                                  0.0s
 => => transferring context: 2B                                                                                                                                                                    0.0s
 => CANCELED [internal] load build context                                                                                                                                                         0.9s
 => => transferring context: 86.96MB                                                                                                                                                               0.8s
 => CACHED [pytorch_image 1/1] FROM nvcr.io/nvidia/pytorch:24.11-py3@sha256:832d94d665baebc0f3f6cfa3168f21d75f7aa8f02ade5ebb65e102e55df79243                                                       0.0s
 => CACHED [install_dependencies  1/30] FROM nvcr.io/nvidia/tritonserver:24.11-py3-min@sha256:e7c402828ed207928d78bf156ff1931112da97ed19030adac1cd37b51e258b40                                     0.0s
 => CACHED [install_dependencies  2/30] RUN if [ -n "${CCACHE_REMOTE_STORAGE}" ] ; then       curl -k -L ${CCACHE_URL} -o ccache.tar.gz ;       tar -xzf ccache.tar.gz -C /usr/local --strip-comp  0.0s
 => CACHED [install_dependencies  3/30] COPY --from=pytorch_image /usr/local/lib/lib* /usr/local/lib/                                                                                              0.0s
 => CACHED [install_dependencies  4/30] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/torch /usr/local/lib/python3.12/dist-packages/torch                                      0.0s
 => ERROR [install_dependencies  5/30] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/torch-2.6.0a0+ecf3bae40a.nv25.1.dist-info /usr/local/lib/python3.12/dist-packages/torch-  0.0s
 => CACHED [install_dependencies  6/30] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/torchgen /usr/local/lib/python3.12/dist-packages/torchgen                                0.0s
 => CACHED [install_dependencies  7/30] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/torchvision /usr/local/lib/python3.12/dist-packages/torchvision                          0.0s
 => CACHED [install_dependencies  8/30] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/torchvision-0.20.0a0.dist-info /usr/local/lib/python3.12/dist-packages/torchvision-0.20  0.0s
 => CACHED [install_dependencies  9/30] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/torchvision.libs /usr/local/lib/python3.12/dist-packages/torchvision.libs                0.0s
 => CACHED [install_dependencies 10/30] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/setuptools /usr/local/lib/python3.12/dist-packages/setuptools                            0.0s
 => CACHED [install_dependencies 11/30] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/setuptools-70.3.0.dist-info /usr/local/lib/python3.12/dist-packages/setuptools-70.3.0.d  0.0s
 => CACHED [install_dependencies 12/30] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/functorch /usr/local/lib/python3.12/dist-packages/functorch                              0.0s
 => ERROR [install_dependencies 13/30] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/pytorch_triton-3.1.0+cf34004b8.internal.dist-info /usr/local/lib/python3.12/dist-package  0.0s
 => CACHED [install_dependencies 14/30] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/triton /usr/local/lib/python3.12/dist-packages/triton                                    0.0s
 => CACHED [install_dependencies 15/30] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/jinja2 /usr/local/lib/python3.12/dist-packages/jinja2                                    0.0s
 => CACHED [install_dependencies 16/30] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/jinja2-3.1.4.dist-info /usr/local/lib/python3.12/dist-packages/jinja2-3.1.4.dist-info    0.0s
 => CACHED [install_dependencies 17/30] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/networkx /usr/local/lib/python3.12/dist-packages/networkx                                0.0s
 => CACHED [install_dependencies 18/30] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/networkx-3.4.2.dist-info /usr/local/lib/python3.12/dist-packages/networkx-3.4.2.dist-in  0.0s
 => CACHED [install_dependencies 19/30] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/sympy /usr/local/lib/python3.12/dist-packages/sympy                                      0.0s
 => CACHED [install_dependencies 20/30] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/sympy-1.13.1.dist-info /usr/local/lib/python3.12/dist-packages/sympy-1.13.1.dist-info    0.0s
 => CACHED [install_dependencies 21/30] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/packaging /usr/local/lib/python3.12/dist-packages/packaging                              0.0s
 => CACHED [install_dependencies 22/30] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/packaging-23.2.dist-info /usr/local/lib/python3.12/dist-packages/packaging-23.2.dist-in  0.0s
 => CACHED [install_dependencies 23/30] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/flash_attn /usr/local/lib/python3.12/dist-packages/flash_attn                            0.0s
 => ERROR [install_dependencies 24/30] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/flash_attn-2.4.2.dev3.dist-info /usr/local/lib/python3.12/dist-packages/flash_attn-2.4.2  0.0s
 => CACHED [final_stage  2/38] COPY --from=pytorch_image /usr/local/lib/lib* /usr/local/lib/                                                                                                       0.0s
 => CACHED [final_stage  3/38] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/torch /usr/local/lib/python3.12/dist-packages/torch                                               0.0s
 => ERROR [final_stage  4/38] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/torch-2.6.0a0+ecf3bae40a.nv25.1.dist-info /usr/local/lib/python3.12/dist-packages/torch-2.6.0a0+e  0.0s
 => CACHED [final_stage  5/38] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/torchgen /usr/local/lib/python3.12/dist-packages/torchgen                                         0.0s
 => CACHED [final_stage  6/38] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/torchvision /usr/local/lib/python3.12/dist-packages/torchvision                                   0.0s
 => CACHED [final_stage  7/38] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/torchvision-0.20.0a0.dist-info /usr/local/lib/python3.12/dist-packages/torchvision-0.20.0a0.dist  0.0s
 => CACHED [final_stage  8/38] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/torchvision.libs /usr/local/lib/python3.12/dist-packages/torchvision.libs                         0.0s
 => CACHED [final_stage  9/38] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/setuptools /usr/local/lib/python3.12/dist-packages/setuptools                                     0.0s
 => CACHED [final_stage 10/38] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/setuptools-70.3.0.dist-info /usr/local/lib/python3.12/dist-packages/setuptools-70.3.0.dist-info   0.0s
 => CACHED [final_stage 11/38] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/functorch /usr/local/lib/python3.12/dist-packages/functorch                                       0.0s
 => ERROR [final_stage 12/38] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/pytorch_triton-3.1.0+cf34004b8.internal.dist-info /usr/local/lib/python3.12/dist-packages/pytorch  0.0s
 => CACHED [final_stage 13/38] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/triton /usr/local/lib/python3.12/dist-packages/triton                                             0.0s
 => CACHED [final_stage 14/38] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/jinja2 /usr/local/lib/python3.12/dist-packages/jinja2                                             0.0s
 => CACHED [final_stage 15/38] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/jinja2-3.1.4.dist-info /usr/local/lib/python3.12/dist-packages/jinja2-3.1.4.dist-info             0.0s
 => CACHED [final_stage 16/38] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/networkx /usr/local/lib/python3.12/dist-packages/networkx                                         0.0s
 => CACHED [final_stage 17/38] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/networkx-3.4.2.dist-info /usr/local/lib/python3.12/dist-packages/networkx-3.4.2.dist-info         0.0s
 => CACHED [final_stage 18/38] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/sympy /usr/local/lib/python3.12/dist-packages/sympy                                               0.0s
 => CACHED [final_stage 19/38] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/sympy-1.13.1.dist-info /usr/local/lib/python3.12/dist-packages/sympy-1.13.1.dist-info             0.0s
 => CACHED [final_stage 20/38] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/packaging /usr/local/lib/python3.12/dist-packages/packaging                                       0.0s
 => CACHED [final_stage 21/38] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/packaging-23.2.dist-info /usr/local/lib/python3.12/dist-packages/packaging-23.2.dist-info         0.0s
 => CACHED [final_stage 22/38] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/flash_attn /usr/local/lib/python3.12/dist-packages/flash_attn                                     0.0s
 => ERROR [final_stage 23/38] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/flash_attn-2.4.2.dev3.dist-info /usr/local/lib/python3.12/dist-packages/flash_attn-2.4.2.dev3.dis  0.0s
------
 > [install_dependencies  5/30] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/torch-2.6.0a0+ecf3bae40a.nv25.1.dist-info /usr/local/lib/python3.12/dist-packages/torch-2.6.0a0+ecf3bae40a.nv25.1.dist-info:
------
------
 > [install_dependencies 13/30] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/pytorch_triton-3.1.0+cf34004b8.internal.dist-info /usr/local/lib/python3.12/dist-packages/pytorch_triton-3.1.0+cf34004b8.internal.dist-info:
------
------
 > [install_dependencies 24/30] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/flash_attn-2.4.2.dev3.dist-info /usr/local/lib/python3.12/dist-packages/flash_attn-2.4.2.dev3.dist-info:
------
------
 > [final_stage  4/38] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/torch-2.6.0a0+ecf3bae40a.nv25.1.dist-info /usr/local/lib/python3.12/dist-packages/torch-2.6.0a0+ecf3bae40a.nv25.1.dist-info:
------
------
 > [final_stage 12/38] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/pytorch_triton-3.1.0+cf34004b8.internal.dist-info /usr/local/lib/python3.12/dist-packages/pytorch_triton-3.1.0+cf34004b8.internal.dist-info:
------
------
 > [final_stage 23/38] COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/flash_attn-2.4.2.dev3.dist-info /usr/local/lib/python3.12/dist-packages/flash_attn-2.4.2.dev3.dist-info:
------
Dockerfile.triton.trt_llm_backend:146
--------------------
 144 |     COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/packaging-23.2.dist-info /usr/local/lib/python3.12/dist-packages/packaging-23.2.dist-info
 145 |     COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/flash_attn /usr/local/lib/python3.12/dist-packages/flash_attn
 146 | >>> COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/flash_attn-2.4.2.dev3.dist-info /usr/local/lib/python3.12/dist-packages/flash_attn-2.4.2.dev3.dist-info
 147 |     COPY --from=pytorch_image /usr/local/lib/python3.12/dist-packages/flash_attn_2_cuda.cpython-312-*-linux-gnu.so /usr/local/lib/python3.12/dist-packages/
 148 |     
--------------------
ERROR: failed to solve: failed to compute cache key: failed to calculate checksum of ref ea805809-c5b8-4847-b0f8-5d1859a5f044::w9qaegrhmle9ejfu888xtb4b3: "/usr/local/lib/python3.12/dist-packages/flash_attn-2.4.2.dev3.dist-info": not found
exit status 1
fatal: destination path 'server' already exists and is not an empty directory.
/home/benjaminbraun_google_com/server/./build.py:1237: SyntaxWarning: invalid escape sequence '\;'
  df += """
/home/benjaminbraun_google_com/server/./build.py:1580: SyntaxWarning: invalid escape sequence '\o'
  df = """
/home/benjaminbraun_google_com/server/./build.py:1779: SyntaxWarning: invalid escape sequence '\p'
  runargs += ["-v", "\\\\.\pipe\docker_engine:\\\\.\pipe\docker_engine"]
usage: build.py [-h] [-q | -v] [--dryrun] [--no-container-build] [--no-container-interactive] [--no-container-pull] [--container-memory CONTAINER_MEMORY] [--target-platform TARGET_PLATFORM]
                [--target-machine TARGET_MACHINE] [--build-id BUILD_ID] [--build-sha BUILD_SHA] [--build-dir BUILD_DIR] [--install-dir INSTALL_DIR] [--cmake-dir CMAKE_DIR] [--tmp-dir TMP_DIR]
                [--library-paths LIBRARY_PATHS] [--build-type BUILD_TYPE] [-j BUILD_PARALLEL] [--github-organization GITHUB_ORGANIZATION] [--version VERSION] [--container-version CONTAINER_VERSION]
                [--upstream-container-version UPSTREAM_CONTAINER_VERSION] [--container-prebuild-command CONTAINER_PREBUILD_COMMAND] [--no-container-source] [--image IMAGE] [--enable-all]
                [--enable-logging] [--enable-stats] [--enable-metrics] [--enable-gpu-metrics] [--enable-cpu-metrics] [--enable-tracing] [--enable-nvtx] [--enable-gpu] [--enable-mali-gpu]
                [--min-compute-capability MIN_COMPUTE_CAPABILITY] [--endpoint ENDPOINT] [--filesystem FILESYSTEM] [--no-core-build] [--backend BACKEND] [--repo-tag REPO_TAG] [--repoagent REPOAGENT]
                [--cache CACHE] [--no-force-clone] [--extra-core-cmake-arg EXTRA_CORE_CMAKE_ARG] [--override-core-cmake-arg OVERRIDE_CORE_CMAKE_ARG]
                [--extra-backend-cmake-arg EXTRA_BACKEND_CMAKE_ARG] [--override-backend-cmake-arg OVERRIDE_BACKEND_CMAKE_ARG]
build.py: error: unrecognized arguments:  
(my-env) benjaminbraun_google_com@triton-orca-refactor:~/tensorrtllm_backend$
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@BenjaminBraunDev