You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: site-src/guides/index.md
+1-1
Original file line number
Diff line number
Diff line change
@@ -24,7 +24,7 @@ This quickstart guide is intended for engineers familiar with k8s and model serv
24
24
25
25
#### GPU-Based Model Server
26
26
27
-
For this setup, you will need 3 GPUs to run the sample model server. Adjust the number of replicas in `./config/manifests/vllm/deployment.yaml` as needed.
27
+
For this setup, you will need 3 GPUs to run the sample model server. Adjust the number of replicas in `./config/manifests/vllm/gpu-deployment.yaml` as needed.
28
28
Create a Hugging Face secret to download the model [meta-llama/Llama-2-7b-hf](https://huggingface.co/meta-llama/Llama-2-7b-hf). Ensure that the token grants access to this model.
29
29
Deploy a sample vLLM deployment with the proper protocol to work with the LLM Instance Gateway.
0 commit comments