Skip to content

Commit deeb215

Browse files
authored
doc: example pull request
1 parent 36b74d0 commit deeb215

File tree

1 file changed

+16
-0
lines changed

1 file changed

+16
-0
lines changed

source/recipe.yaml

+16
Original file line numberDiff line numberDiff line change
@@ -394,3 +394,19 @@
394394
extra_labels:
395395
openllm_alias: 3.8b-q4,3.8b-mini-q4,3.8b-mini-instruct-4k-ggml-q4
396396
openllm_hf_model_id: microsoft/Phi-3-mini-4k-instruct-gguf
397+
"qwen2:72b-instruct-fp16":
398+
project: vllm-chat
399+
service_config:
400+
name: qwen2
401+
traffic:
402+
timeout: 300
403+
resources:
404+
gpu: 2
405+
gpu_type: nvidia-a100-80g
406+
engine_config:
407+
model: Qwen/Qwen2-72B-Instruct
408+
max_model_len: 2048
409+
dtype: half
410+
extra_labels:
411+
openllm_alias: 72b,72b-instruct
412+
openllm_hf_model_id: Qwen/Qwen2-72B-Instruct

0 commit comments

Comments
 (0)