Skip to content

Commit d0d01e1

Browse files
committed
Add config documentation
Signed-off-by: Kunjan Patel <[email protected]>
1 parent c5d2527 commit d0d01e1

File tree

2 files changed

+24
-4
lines changed

2 files changed

+24
-4
lines changed

examples/dynamic-lora-sidecar/README.md

+19
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,25 @@ The sidecar uses the vLLM server's API to load or unload adapters based on the c
4545

4646
[deployment]: deployment.yaml
4747

48+
## Configuration Fields
49+
- `vLLMLoRAConfig`[**required**] base key
50+
- `host` [*optional*]Model server's host. defaults to localhost
51+
- `port` [*optional*] Model server's port. defaults to 8000
52+
- `name`[*optional*] Name of this config
53+
- `ensureExist`[*optional*] List of models to ensure existence on specified model server.
54+
- `models`[**required**] [list]
55+
- `base-model`[*optional*] Base model for lora adapter
56+
- `id`[**required**] unique id of lora adapter
57+
- `source`[**required**] path (remote or local) to lora adapter
58+
- `ensureNotExist` [*optional*]
59+
- `models`[**required**] [list]
60+
- `id`[**required**] unique id of lora adapter
61+
- `source`[**required**] path (remote or local) to lora adapter
62+
- `base-model`[*optional*] Base model for lora adapter
63+
64+
65+
66+
4867
## Screenshots & Testing
4968
I tested the sidecar in my cluster with deployment and configmap specified in this repo. Here are the screen grabs of the logs from the sidecar and vllm server. I used the specified configmap, verified that the adapters were loaded by querying `v1/models` and looking at vllm logs. I changed the configmap and validated the same on vllm server. Note: There is slight lag between updates.
5069
![lora-adapter-syncer](screenshots/lora-syncer-sidecar.png)
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
1-
aiohttp==3.10.10
2-
pyyaml==6.0.2
3-
requests==2.32.3
4-
watchfiles==0.24.0
1+
aiohttp
2+
jsonschema
3+
pyyaml
4+
requests
5+
watchfiles

0 commit comments

Comments
 (0)