Skip to content

Commit fb43dee

Browse files
hongxiayangtjtanaa
authored andcommitted
[Documentation][AMD] Add information about prebuilt ROCm vLLM docker for perf validation purpose (vllm-project#12281)
Signed-off-by: Hongxia Yang <[email protected]>
1 parent 10611d8 commit fb43dee

File tree

1 file changed

+8
-0
lines changed
  • docs/source/getting_started/installation/gpu

1 file changed

+8
-0
lines changed

docs/source/getting_started/installation/gpu/rocm.inc.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,14 @@ vLLM supports AMD GPUs with ROCm 6.3.
1313

1414
Currently, there are no pre-built ROCm wheels.
1515

16+
However, the [AMD Infinity hub for vLLM](https://hub.docker.com/r/rocm/vllm/tags) offers a prebuilt, optimized
17+
docker image designed for validating inference performance on the AMD Instinct™ MI300X accelerator.
18+
19+
```{tip}
20+
Please check [LLM inference performance validation on AMD Instinct MI300X](https://rocm.docs.amd.com/en/latest/how-to/performance-validation/mi300x/vllm-benchmark.html)
21+
for instructions on how to use this prebuilt docker image.
22+
```
23+
1624
### Build wheel from source
1725

1826
0. Install prerequisites (skip if you are already in an environment/docker with the following installed):

0 commit comments

Comments
 (0)