Allow defining a default base model in the lora syncer configuration

Author: kaushikmitr

tools/dynamic-lora-sidecar/README.md

@@ -60,20 +60,67 @@ The sidecar supports the following command-line arguments:
 
 ## Configuration Fields
 - `vLLMLoRAConfig`[**required**]  base key 
-- `host` [*optional*]Model server's host. defaults to localhost
+- `host` [*optional*] Model server's host. defaults to localhost
 - `port` [*optional*] Model server's port. defaults to 8000
-- `name`[*optional*] Name of this config
-- `ensureExist`[*optional*] List of models to ensure existence on specified model server.
-    -  `models`[**required**] [list]
-        - `base-model`[*optional*] Base model for lora adapter
-        - `id`[**required**] unique id of lora adapter
-        - `source`[**required**] path (remote or local) to lora adapter
+- `name` [*optional*] Name of this config
+- `defaultBaseModel` [*optional*] Default base model to use for all adapters when not specified individually
+- `ensureExist` [*optional*] List of models to ensure existence on specified model server.
+    -  `models` [**required**] [list]
+        - `id` [**required**] unique id of lora adapter
+        - `source` [**required**] path (remote or local) to lora adapter
+        - `base-model` [*optional*] Base model for lora adapter (overrides defaultBaseModel)
 - `ensureNotExist` [*optional*]
-    - `models`[**required**] [list]
-        - `id`[**required**] unique id of lora adapter
-        -  `source`[**required**] path (remote or local) to lora adapter
-        - `base-model`[*optional*] Base model for lora adapter
+    - `models` [**required**] [list]
+        - `id` [**required**] unique id of lora adapter
+        - `source` [**required**] path (remote or local) to lora adapter
+        - `base-model` [*optional*] Base model for lora adapter (overrides defaultBaseModel)
 
+## Example Configuration
+
+Here's an example of using the `defaultBaseModel` field to avoid repetition in your configuration:
+
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: vllm-llama2-7b-adapters
+data:
+  configmap.yaml: |
+      vLLMLoRAConfig:
+        name: vllm-llama2-7b
+        port: 8000
+        defaultBaseModel: meta-llama/Llama-2-7b-hf
+        ensureExist:
+          models:
+          - id: tweet-summary-1
+            source: vineetsharma/qlora-adapter-Llama-2-7b-hf-TweetSumm
+          - id: tweet-summary-2
+            source: mahimairaja/tweet-summarization-llama-2-finetuned  
+```
+
+In this example, both adapters will use `meta-llama/Llama-2-7b-hf` as their base model without needing to specify it for each adapter individually.
+
+You can still override the default base model for specific adapters when needed:
+
+```yaml
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: vllm-mixed-adapters
+data:
+  configmap.yaml: |
+      vLLMLoRAConfig:
+        name: vllm-mixed
+        port: 8000
+        defaultBaseModel: meta-llama/Llama-2-7b-hf
+        ensureExist:
+          models:
+          - id: tweet-summary-1
+            source: vineetsharma/qlora-adapter-Llama-2-7b-hf-TweetSumm
+          - id: code-assistant
+            source: huggingface/code-assistant-lora
+            base-model: meta-llama/Llama-2-13b-hf  # Override for this specific adapter
+```
 ## Example Deployment
 
 The [deployment.yaml](deployment.yaml) file shows an example of deploying the sidecar with custom parameters:

tools/dynamic-lora-sidecar/sidecar/sidecar.py

@@ -135,15 +135,24 @@ def port(self):
     def model_server(self):
         """Model server {host}:{port}"""
         return f"{self.host}:{self.port}"
+    
+    @property
+    def default_base_model(self):
+        """Default base model to use when not specified at adapter level"""
+        return self.config.get("defaultBaseModel", "")
 
     @property
     def ensure_exist_adapters(self):
         """Lora adapters in config under key `ensureExist` in set"""
         adapters = self.config.get("ensureExist", {}).get("models", set())
+        default_model = self.default_base_model
+        
         return set(
             [
                 LoraAdapter(
-                    adapter["id"], adapter["source"], adapter.get("base-model", "")
+                    adapter["id"], 
+                    adapter["source"], 
+                    adapter.get("base-model", default_model)
                 )
                 for adapter in adapters
             ]
@@ -153,10 +162,14 @@ def ensure_exist_adapters(self):
     def ensure_not_exist_adapters(self):
         """Lora adapters in config under key `ensureNotExist` in set"""
         adapters = self.config.get("ensureNotExist", {}).get("models", set())
+        default_model = self.default_base_model
+        
         return set(
             [
                 LoraAdapter(
-                    adapter["id"], adapter["source"], adapter.get("base-model", "")
+                    adapter["id"], 
+                    adapter["source"], 
+                    adapter.get("base-model", default_model)
                 )
                 for adapter in adapters
             ]

tools/dynamic-lora-sidecar/sidecar/validation.yaml

@@ -16,6 +16,9 @@ properties:
       name:
         type: string
         description: Name of this config
+      defaultBaseModel:
+        type: string
+        description: Default base model to use when not specified at adapter level
       ensureExist:
         type: object
         description: List of models to ensure existence on specified model server
@@ -26,9 +29,9 @@ properties:
             items:
               type: object
               properties:
-                base_model:
+                base-model:
                   type: string
-                  description: Base model for LoRA adapter
+                  description: Base model for LoRA adapter (overrides defaultBaseModel)
                 id:
                   type: string
                   description: Unique ID of LoRA adapter
@@ -50,9 +53,9 @@ properties:
             items:
               type: object
               properties:
-                base_model:
+                base-model:
                   type: string
-                  description: Base model for LoRA adapter
+                  description: Base model for LoRA adapter (overrides defaultBaseModel)
                 id:
                   type: string
                   description: Unique ID of LoRA adapter