Skip to content

Commit d8bfad9

Browse files
youkaichaosumitd2
authored andcommitted
[doc] organize installation doc and expose per-commit docker (vllm-project#8931)
Signed-off-by: Sumit Dubey <[email protected]>
1 parent 958d46c commit d8bfad9

File tree

1 file changed

+24
-12
lines changed

1 file changed

+24
-12
lines changed

docs/source/getting_started/installation.rst

Lines changed: 24 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -12,8 +12,8 @@ Requirements
1212
* Python: 3.8 -- 3.12
1313
* GPU: compute capability 7.0 or higher (e.g., V100, T4, RTX20xx, A100, L4, H100, etc.)
1414

15-
Install with pip
16-
----------------
15+
Install released versions
16+
--------------------------
1717

1818
You can install vLLM using pip:
1919

@@ -46,22 +46,34 @@ You can install vLLM using pip:
4646

4747
Therefore, it is recommended to install vLLM with a **fresh new** conda environment. If either you have a different CUDA version or you want to use an existing PyTorch installation, you need to build vLLM from source. See below for instructions.
4848

49-
.. note::
49+
Install the latest code
50+
----------------------------
5051

51-
vLLM also publishes wheels for Linux running on x86 platform with cuda 12 for every commit since v0.5.3. You can download and install them with the following command:
52+
LLM inference is a fast-evolving field, and the latest code may contain bug fixes, performance improvements, and new features that are not released yet. To allow users to try the latest code without waiting for the next release, vLLM provides wheels for Linux running on x86 platform with cuda 12 for every commit since v0.5.3. You can download and install the latest one with the following command:
5253

53-
.. code-block:: console
54+
.. code-block:: console
5455
55-
$ export VLLM_COMMIT=33f460b17a54acb3b6cc0b03f4a17876cff5eafd # use full commit hash from the main branch
56-
$ pip install https://vllm-wheels.s3.us-west-2.amazonaws.com/${VLLM_COMMIT}/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl
56+
$ pip install https://vllm-wheels.s3.us-west-2.amazonaws.com/nightly/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl
5757
58-
You can also just download the latest wheel by running:
58+
If you want to access the wheels for previous commits, you can specify the commit hash in the URL:
5959

60-
.. code-block:: console
60+
.. code-block:: console
61+
62+
$ export VLLM_COMMIT=33f460b17a54acb3b6cc0b03f4a17876cff5eafd # use full commit hash from the main branch
63+
$ pip install https://vllm-wheels.s3.us-west-2.amazonaws.com/${VLLM_COMMIT}/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl
64+
65+
Note that the wheels are built with Python 3.8 abi (see `PEP 425 <https://peps.python.org/pep-0425/>`_ for more details about abi), so **they are compatible with Python 3.8 and later**. The version string in the wheel file name (``1.0.0.dev``) is just a placeholder to have a unified URL for the wheels. The actual versions of wheels are contained in the wheel metadata.
66+
67+
Another way to access the latest code is to use the docker images:
68+
69+
.. code-block:: console
70+
71+
$ export VLLM_COMMIT=33f460b17a54acb3b6cc0b03f4a17876cff5eafd # use full commit hash from the main branch
72+
$ docker pull public.ecr.aws/q9t5s3a7/vllm-ci-test-repo:${VLLM_COMMIT}
6173
62-
$ pip install https://vllm-wheels.s3.us-west-2.amazonaws.com/nightly/vllm-1.0.0.dev-cp38-abi3-manylinux1_x86_64.whl
74+
These docker images are used for CI and testing only, and they are not intended for production use. They will be expired after several days.
6375

64-
Note that the wheels are built with Python 3.8 abi (see `PEP 425 <https://peps.python.org/pep-0425/>`_ for more details about abi), so they are compatible with Python 3.8 and later. The version string in the wheel file name (``1.0.0.dev``) is just a placeholder to have a unified URL for the wheels. The actual version of wheels is contained in the wheel metadata.
76+
Latest code can contain bugs and may not be stable. Please use it with caution.
6577

6678
Build from source (without compilation)
6779
---------------------------------------
@@ -102,7 +114,7 @@ If you need to touch the C++ or CUDA code, you need to build vLLM from source:
102114
103115
$ git clone https://github.com/vllm-project/vllm.git
104116
$ cd vllm
105-
$ pip install -e . # This may take 5-10 minutes.
117+
$ pip install -e . # This can take a long time
106118
107119
.. note::
108120

0 commit comments

Comments
 (0)