OLS-1417: Multiple RAG index

[raptorsun]

Description

This PR works together with this PR on lightspeed-service side.

This PR adds operator support for multiple RAG indexes.

A Custom RAG option is added into OLSConfig CRD. RAG is configured by image, indexPath in that image and IndexID to search on.

apiVersion: ols.openshift.io/v1alpha1
kind: OLSConfig
metadata:
  name: cluster
spec:
  ols:
    rag:
      - image: quay.io/redhat-user-workloads/crt-nshift-lightspeed-tenant/ols/lightspeed-rag-content@sha256:e4420ab37f9ad90daeab93ba6717db6d388c69a357d0eeb486d1d2d0bc15e085
        indexPath: /rag/vector_db/ocp_product_docs/4.15
        indexID: ocp-product-docs-4_15

The operator can download RAG image and extract index from the indexPath of the image and put into a shared volume to be accessed by the service API server.

Here is the RAG content copied to the shared volume in service API pod.

sh-5.1$ ls -R 
.:
rag-0

./rag-0:
4.15

./rag-0/4.15:
default__vector_store.json  docstore.json  graph_store.json  image__vector_store.json  index_store.json  metadata.json

Type of change

• Refactor
• New feature
• Bug fix
• CVE fix
• Optimization
• Documentation Update
• Configuration Update
• Bump-up dependent library

Related Tickets & Documents

• Related Issue # OLS-1418
• Closes # OLS-1417

Checklist before requesting a review

• I have performed a self-review of my code.
• PR has passed all pre-merge test jobs.
• If it is a core feature, I have added thorough tests.

Testing

• Please provide detailed steps to perform tests related to this code change.
• How were the fix/results from this change verified? Please provide relevant screenshots or results.

[openshift-ci-robot]

@raptorsun: This pull request references OLS-1417 which is a valid jira issue.

In response to this:

Description

This PR works together with this PR on lightspeed-service side.

This PR adds operator support for multiple RAG indexes.

A Custom RAG option is added into OLSConfig CRD. RAG is configured by image, indexPath in that image and IndexID to search on.

apiVersion: ols.openshift.io/v1alpha1
kind: OLSConfig
metadata:
 name: cluster
spec:
 ols:
   rag:
     - image: quay.io/redhat-user-workloads/crt-nshift-lightspeed-tenant/ols/lightspeed-rag-content@sha256:e4420ab37f9ad90daeab93ba6717db6d388c69a357d0eeb486d1d2d0bc15e085
       indexPath: /rag/vector_db/ocp_product_docs/4.15
       indexID: ocp-product-docs-4_15

The operator can download RAG image and extract index from the indexPath of the image and put into a shared volume to be accessed by the service API server.

Here is the RAG content copied to the shared volume in service API pod.

sh-5.1$ ls -R 
.:
rag-0

./rag-0:
4.15

./rag-0/4.15:
default__vector_store.json  docstore.json  graph_store.json  image__vector_store.json  index_store.json  metadata.json

Type of change

• Refactor
• New feature
• Bug fix
• CVE fix
• Optimization
• Documentation Update
• Configuration Update
• Bump-up dependent library

Related Tickets & Documents

• Related Issue # OLS-1418
• Closes # OLS-1417

Checklist before requesting a review

• I have performed a self-review of my code.
• PR has passed all pre-merge test jobs.
• If it is a core feature, I have added thorough tests.

Testing

• Please provide detailed steps to perform tests related to this code change.
• How were the fix/results from this change verified? Please provide relevant screenshots or results.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci-robot]

@raptorsun: This pull request references OLS-1417 which is a valid jira issue.

In response to this:

Description

This PR works together with this PR on lightspeed-service side.

This PR adds operator support for multiple RAG indexes.

A Custom RAG option is added into OLSConfig CRD. RAG is configured by image, indexPath in that image and IndexID to search on.

apiVersion: ols.openshift.io/v1alpha1
kind: OLSConfig
metadata:
 name: cluster
spec:
 ols:
   rag:
     - image: quay.io/redhat-user-workloads/crt-nshift-lightspeed-tenant/ols/lightspeed-rag-content@sha256:e4420ab37f9ad90daeab93ba6717db6d388c69a357d0eeb486d1d2d0bc15e085
       indexPath: /rag/vector_db/ocp_product_docs/4.15
       indexID: ocp-product-docs-4_15

The operator can download RAG image and extract index from the indexPath of the image and put into a shared volume to be accessed by the service API server.

Here is the RAG content copied to the shared volume in service API pod.

sh-5.1$ ls -R 
.:
rag-0

./rag-0:
4.15

./rag-0/4.15:
default__vector_store.json  docstore.json  graph_store.json  image__vector_store.json  index_store.json  metadata.json

Type of change

• Refactor
• New feature
• Bug fix
• CVE fix
• Optimization
• Documentation Update
• Configuration Update
• Bump-up dependent library

Related Tickets & Documents

• Related Issue # OLS-1418
• Closes # OLS-1417

Checklist before requesting a review

• I have performed a self-review of my code.
• PR has passed all pre-merge test jobs.
• If it is a core feature, I have added thorough tests.

Testing

• Please provide detailed steps to perform tests related to this code change.
• How were the fix/results from this change verified? Please provide relevant screenshots or results.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign raptorsun for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[openshift-ci]

@raptorsun: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/bundle-e2e-4-17	cb91090	link	true	/test bundle-e2e-4-17

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[openshift-merge-robot]

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.