You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When I install vLLM, I have pull the docker as the tutorial and install from source. But there are an error encountered, how can I fix this: ValueError: Unsupported CUDA arch (5.2). Valid CUDA arch strings are: ['7.0', '7.5', '8.0', '8.6', '8.9', '9.0', '7.0+PTX', '7.5+PTX', '8.0+PTX', '8.6+PTX', '8.9+PTX', '9.0+PTX'].
The text was updated successfully, but these errors were encountered:
I managed to get it working by setting ENV TORCH_CUDA_ARCH_LIST="8.0;8.6;8.9" (depends on your GPU) but I think an ENV TORCH_CUDA_ARCH_LIST="" could also work.
When I install vLLM, I have pull the docker as the tutorial and install from source. But there are an error encountered, how can I fix this:
ValueError: Unsupported CUDA arch (5.2). Valid CUDA arch strings are: ['7.0', '7.5', '8.0', '8.6', '8.9', '9.0', '7.0+PTX', '7.5+PTX', '8.0+PTX', '8.6+PTX', '8.9+PTX', '9.0+PTX'].
The text was updated successfully, but these errors were encountered: