Install vLLM failed with pip install -e ., PyTorch dependency confusion?

[gesanqiu]

Hi guys, I install vllm failed with pip install -e .
The error messages shows below:

(vllm) dell@dell:~/workSpace/vllm$ pip install -e .
Looking in indexes: https://pypi.tuna.tsinghua.edu.cn/simple
Obtaining file:///home/dell/workSpace/vllm
  Installing build dependencies ... done
  Checking if build backend supports build_editable ... done
  Getting requirements to build editable ... error
  error: subprocess-exited-with-error

  × Getting requirements to build editable did not run successfully.
  │ exit code: 1
  ╰─> [28 lines of output]
      /tmp/pip-build-env-6fb626c8/overlay/lib/python3.10/site-packages/torch/nn/modules/transformer.py:20: UserWarning: Failed to initialize NumPy: No module named 'numpy' (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:84.)
        device: torch.device = torch.device(torch._C._get_default_device()),  # torch.device('cpu'),
      /tmp/pip-build-env-6fb626c8/overlay/lib/python3.10/site-packages/torch/cuda/__init__.py:138: UserWarning: CUDA initialization: The NVIDIA driver on your system is too old (found version 11080). Please update your GPU driver by downloading and installing a new version from the URL: http://www.nvidia.com/Download/index.aspx Alternatively, go to: https://pytorch.org to install a PyTorch version that has been compiled with your version of the CUDA driver. (Triggered internally at ../c10/cuda/CUDAFunctions.cpp:108.)
        return torch._C._cuda_getDeviceCount() > 0
      No CUDA runtime is found, using CUDA_HOME='/usr/local/cuda'

I have no idea about this issue, even though I pre-install pytorch=2.0.1+cu11.8, it still failed.

My driver version is 520.61.05 and cuda version is 11.8. While I can't install vllm in an editable way, but installation is success with pip install vllm.
After I update my driver to 535.104.12(latest version), I got another error messages:

  error: subprocess-exited-with-error

  × Building editable for vllm (pyproject.toml) did not run successfully.
  │ exit code: 1
  ╰─> [125 lines of output]
      /tmp/pip-build-env-id48fazi/overlay/lib/python3.10/site-packages/torch/nn/modules/transformer.py:20: UserWarning: Failed to initialize NumPy: No module named 'numpy' (Triggered internally at ../torch/csrc/utils/tensor_numpy.cpp:84.)
        device: torch.device = torch.device(torch._C._get_default_device()),  # torch.device('cpu'),
      running editable_wheel
      creating /tmp/pip-wheel-ynbd83yq/.tmp-3612t8tz/vllm.egg-info
      writing /tmp/pip-wheel-ynbd83yq/.tmp-3612t8tz/vllm.egg-info/PKG-INFO
      writing dependency_links to /tmp/pip-wheel-ynbd83yq/.tmp-3612t8tz/vllm.egg-info/dependency_links.txt
      writing requirements to /tmp/pip-wheel-ynbd83yq/.tmp-3612t8tz/vllm.egg-info/requires.txt
      writing top-level names to /tmp/pip-wheel-ynbd83yq/.tmp-3612t8tz/vllm.egg-info/top_level.txt
      writing manifest file '/tmp/pip-wheel-ynbd83yq/.tmp-3612t8tz/vllm.egg-info/SOURCES.txt'
      reading manifest file '/tmp/pip-wheel-ynbd83yq/.tmp-3612t8tz/vllm.egg-info/SOURCES.txt'
      reading manifest template 'MANIFEST.in'
      adding license file 'LICENSE'
      writing manifest file '/tmp/pip-wheel-ynbd83yq/.tmp-3612t8tz/vllm.egg-info/SOURCES.txt'
      creating '/tmp/pip-wheel-ynbd83yq/.tmp-3612t8tz/vllm-0.2.0.dist-info'
      creating /tmp/pip-wheel-ynbd83yq/.tmp-3612t8tz/vllm-0.2.0.dist-info/WHEEL
      running build_py
      running build_ext
      Traceback (most recent call last):
        File "/tmp/pip-build-env-id48fazi/overlay/lib/python3.10/site-packages/setuptools/command/editable_wheel.py", line 156, in run
          self._create_wheel_file(bdist_wheel)
        File "/tmp/pip-build-env-id48fazi/overlay/lib/python3.10/site-packages/setuptools/command/editable_wheel.py", line 345, in _create_wheel_file
          files, mapping = self._run_build_commands(dist_name, unpacked, lib, tmp)
        File "/tmp/pip-build-env-id48fazi/overlay/lib/python3.10/site-packages/setuptools/command/editable_wheel.py", line 268, in _run_build_commands
          self._run_build_subcommands()
        File "/tmp/pip-build-env-id48fazi/overlay/lib/python3.10/site-packages/setuptools/command/editable_wheel.py", line 295, in _run_build_subcommands
          self.run_command(name)
        File "/tmp/pip-build-env-id48fazi/overlay/lib/python3.10/site-packages/setuptools/_distutils/cmd.py", line 318, in run_command
          self.distribution.run_command(command)
        File "/tmp/pip-build-env-id48fazi/overlay/lib/python3.10/site-packages/setuptools/dist.py", line 989, in run_command
          super().run_command(command)
        File "/tmp/pip-build-env-id48fazi/overlay/lib/python3.10/site-packages/setuptools/_distutils/dist.py", line 988, in run_command
          cmd_obj.run()
        File "/tmp/pip-build-env-id48fazi/overlay/lib/python3.10/site-packages/setuptools/command/build_ext.py", line 88, in run
          _build_ext.run(self)
        File "/tmp/pip-build-env-id48fazi/overlay/lib/python3.10/site-packages/setuptools/_distutils/command/build_ext.py", line 345, in run
          self.build_extensions()
        File "/tmp/pip-build-env-id48fazi/overlay/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 525, in build_extensions
          _check_cuda_version(compiler_name, compiler_version)
        File "/tmp/pip-build-env-id48fazi/overlay/lib/python3.10/site-packages/torch/utils/cpp_extension.py", line 413, in _check_cuda_version
          raise RuntimeError(CUDA_MISMATCH_MESSAGE.format(cuda_str_version, torch.version.cuda))
      RuntimeError:
      The detected CUDA version (11.8) mismatches the version that was used to compile
      PyTorch (12.1). Please make sure to use the same CUDA versions.

It's weird that vLLM should not support CUDA 12.1, in which case it will depend on CUDA 12.1?
And after install the CUDA 12.2, I install vLLM succeed but can't run.

Traceback (most recent call last):
  File "/home/dell/workSpace/vllm/vllm/entrypoints/openai/api_server.py", line 616, in <module>
    engine = AsyncLLMEngine.from_engine_args(engine_args)
  File "/home/dell/workSpace/vllm/vllm/engine/async_llm_engine.py", line 487, in from_engine_args
    engine = cls(engine_args.worker_use_ray,
  File "/home/dell/workSpace/vllm/vllm/engine/async_llm_engine.py", line 270, in __init__
    self.engine = self._init_engine(*args, **kwargs)
  File "/home/dell/workSpace/vllm/vllm/engine/async_llm_engine.py", line 306, in _init_engine
    return engine_class(*args, **kwargs)
  File "/home/dell/workSpace/vllm/vllm/engine/llm_engine.py", line 110, in __init__
    self._init_workers(distributed_init_method)
  File "/home/dell/workSpace/vllm/vllm/engine/llm_engine.py", line 128, in _init_workers
    from vllm.worker.worker import Worker  # pylint: disable=import-outside-toplevel
  File "/home/dell/workSpace/vllm/vllm/worker/worker.py", line 10, in <module>
    from vllm.model_executor import get_model, InputMetadata, set_random_seed
  File "/home/dell/workSpace/vllm/vllm/model_executor/__init__.py", line 2, in <module>
    from vllm.model_executor.model_loader import get_model
  File "/home/dell/workSpace/vllm/vllm/model_executor/model_loader.py", line 10, in <module>
    from vllm.model_executor.models import *  # pylint: disable=wildcard-import
  File "/home/dell/workSpace/vllm/vllm/model_executor/models/__init__.py", line 1, in <module>
    from vllm.model_executor.models.aquila import AquilaForCausalLM
  File "/home/dell/workSpace/vllm/vllm/model_executor/models/aquila.py", line 35, in <module>
    from vllm.model_executor.layers.attention import PagedAttentionWithRoPE
  File "/home/dell/workSpace/vllm/vllm/model_executor/layers/attention.py", line 10, in <module>
    from vllm import attention_ops
ImportError: /home/dell/workSpace/vllm/vllm/attention_ops.cpython-38-x86_64-linux-gnu.so: undefined symbol: _ZNK3c1010TensorImpl27throw_data_ptr_access_errorEv

I also tried install vllm in nvidia/cuda11.8.0-cudnn8-devel-ubuntu20.04 docker inage, got the same issue.

[fengyizhu]

+1

[dachengai]

+1

[fengyizhu]

Due to the release of PyTorch 2.1.0, the torch version has been locked to 2.0.1 in the pyproject.toml file.

[build-system]
requires = [
    "ninja",
    "packaging",
    "setuptools",
    "torch == 2.0.1",
    "wheel",
]
build-backend = "setuptools.build_meta"

[tendaysaweek]

+1

[elkay]

Landed here with the same errors as OP. What version of torch is required now for vLLM? I'm trying to use vLLM built from source and currently on 2.1.2.

[davidfrankenberg]

+1