Merge pull request #38 from hemajv/add-milvus

Update llamastack to use in-line milvus

Author: cooktheryan

kubernetes/README.md

@@ -78,6 +78,43 @@ podman tag localhost/distribution-remote-vllm:dev quay.io/redhat-et/llama:2-27-2
 podman push quay.io/redhat-et/llama:2-27-2025
 ```
 
+## Building Llamastack with Milvus (in-line)
+
+If you need to build Llamastack to use Milvus as the default in-line vector db provider in your container image, you can run the following steps:
+
+```
+git clone [email protected]:meta-llama/llama-stack.git
+cd llama-stack
+
+# Create the venv to install llamastack locally
+python -m venv venv
+source venv/bin/activate
+pip install -U .
+```
+
+Edit the `build.yaml` in the `llama_stack/template/remote-vllm/build.yaml` to update the `vector_io` provider field and `image_type` field as shown below:
+
+```
+  providers:
+    vector_io:
+    - inline::milvus
+image_type: container
+```
+
+Now, we can build the container.
+
+```
+export CONTAINER_BINARY = podman
+USE_COPY_NOT_MOUNT=true LLAMA_STACK_DIR=. llama stack build --config llama_stack/templates/remote-vllm/build.yaml --image-type container --image-name remote-vllm-milvus
+```
+
+Once the image is built successfully you can push it to quay:
+
+```
+podman tag localhost/remote-vllm-milvus:<version tag> quay.io/<quay user name or org name>/<image name>
+podman push quay.io/<quay user name or org/<image name>
+```
+
 Update the [`deployment.yaml`](https://github.com/redhat-et/agent-frameworks/blob/main/prototype/frameworks/llamastack/kubernetes/llama-stack/deployment.yaml#L28) file using the image generated above.
 
 ## Configmap

kubernetes/llama-stack/configmap.yaml

@@ -36,13 +36,10 @@ data:
         provider_type: inline::sentence-transformers
         config: {}
       vector_io:
-      - provider_id: faiss
-        provider_type: inline::faiss
+      - provider_id: milvus
+        provider_type: inline::milvus
         config:
-          kvstore:
-            type: sqlite
-            namespace: null
-            db_path: ${env.SQLITE_STORE_DIR:~/.llama/distributions/remote-vllm}/faiss_store.db
+          db_path: ${env.MILVUS_DB_PATH}
       safety:
       - provider_id: llama-guard
         provider_type: inline::llama-guard

kubernetes/llama-stack/deployment.yaml

@@ -33,7 +33,9 @@ spec:
           value: http://otel-collector-collector.observability-hub.svc.cluster.local:4318/v1/traces
         - name: OTEL_METRIC_ENDPOINT
           value: http://otel-collector-collector.observability-hub.svc.cluster.local:4318/v1/metrics
-        image: llamastack/distribution-remote-vllm:0.1.8
+        - name: MILVUS_DB_PATH
+          value: 'milvus.db'
+        image: quay.io/redhat-et/llama:vllm-0.1.9
         imagePullPolicy: Always
         name: llamastack
         ports: