Skip to content

Dynamic lora load/unload sidecar #31

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 34 commits into from
Nov 18, 2024
Merged

Dynamic lora load/unload sidecar #31

merged 34 commits into from
Nov 18, 2024

Conversation

coolkp
Copy link
Contributor

@coolkp coolkp commented Oct 23, 2024

Adding sidecar example for dynamically managing lora adapters on vllm server

@linux-foundation-easycla Linux Foundation: EasyCLA
Copy link

linux-foundation-easycla bot commented Oct 23, 2024 •
edited
Loading

CLA Signed

The committers listed above are authorized under a signed CLA.

@k8s-ci-robot k8s-ci-robot added cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. and removed cncf-cla: no Indicates the PR's author has not signed the CNCF CLA. labels Oct 23, 2024
@liu-cong
Copy link
Contributor

/assign

liu-cong
liu-cong reviewed Oct 24, 2024
View reviewed changes
guydc
guydc reviewed Oct 25, 2024
View reviewed changes
@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Oct 30, 2024
@k8s-ci-robot k8s-ci-robot added size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Oct 30, 2024
@coolkp
Copy link
Contributor Author

coolkp commented Oct 30, 2024

Thanks @liu-cong @guydc for reviewing, deep apologies for delay in responding. I had my notifications misconfigured, emails went to wrong place :(

coolkp added 5 commits October 30, 2024 02:48
@coolkp
base model is optional
5a03f98 
Signed-off-by: Kunjan Patel <[email protected]>
@coolkp
Check health of server before querying
1af2df4 
Signed-off-by: Kunjan Patel <[email protected]>
@coolkp
Check health of server before querying
5b51182 
Signed-off-by: Kunjan Patel <[email protected]>
@coolkp
Docstrings
cc1e686 
Signed-off-by: Kunjan Patel <[email protected]>
@coolkp
Mock health check in tests
926a71c 
Signed-off-by: Kunjan Patel <[email protected]>
liu-cong
liu-cong reviewed Nov 5, 2024
View reviewed changes
@coolkp
Refactor configmap, switch to watchfiles to detect symbolic link targ…
cb3c9b2 
…et changes, pull dynamically from configmap

Signed-off-by: Kunjan Patel <[email protected]>
@coolkp
Add testing results
b30051a 
Signed-off-by: Kunjan Patel <[email protected]>
@coolkp coolkp requested a review from ahg-g November 9, 2024 02:00
ahg-g
ahg-g reviewed Nov 11, 2024
View reviewed changes
Copy link
Contributor

@ahg-g ahg-g left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why are we placing this under examples?

@coolkp
Add config validation
c5d2527 
Signed-off-by: Kunjan Patel <[email protected]>
@coolkp
Add config documentation
d0d01e1 
Signed-off-by: Kunjan Patel <[email protected]>
@coolkp
Add config documentation
b4867b6 
Signed-off-by: Kunjan Patel <[email protected]>
@coolkp
Copy link
Contributor Author

coolkp commented Nov 11, 2024

why are we placing this under examples?
Where do you suggest we place it?
@ahg-g

@ahg-g
Copy link
Contributor

ahg-g commented Nov 12, 2024

why are we placing this under examples?
Where do you suggest we place it?
@ahg-g

perhaps under a tools directory?

@coolkp
Move under tools
c24ff35 
Signed-off-by: Kunjan <[email protected]>
@coolkp
Move under tools
472b545 
Signed-off-by: Kunjan <[email protected]>
@coolkp coolkp requested review from liu-cong and ahg-g November 12, 2024 18:46
ahg-g
ahg-g reviewed Nov 16, 2024
View reviewed changes
coolkp and others added 2 commits November 16, 2024 11:49
@coolkp coolkp requested a review from ahg-g November 16, 2024 19:49
@ahg-g
Copy link
Contributor

ahg-g commented Nov 18, 2024

/lgtm
/approve

/label tide/merge-method-squash

@k8s-ci-robot k8s-ci-robot added the tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges. label Nov 18, 2024
@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Nov 18, 2024
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ahg-g, coolkp

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 18, 2024
@k8s-ci-robot k8s-ci-robot merged commit 54ee6d7 into kubernetes-sigs:main Nov 18, 2024
2 checks passed
shaneutt pushed a commit to shaneutt/gateway-api-inference-extension that referenced this pull request Apr 18, 2025
@clubanderson
Merge pull request kubernetes-sigs#31 from shaneutt/shaneutt/crd-depl…
e7a53af 
…oyments

Add CRDs deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@guydc guydc guydc left review comments

@kfswain kfswain Awaiting requested review from kfswain

@robscott robscott Awaiting requested review from robscott

@liu-cong liu-cong Awaiting requested review from liu-cong

@ahg-g ahg-g Awaiting requested review from ahg-g

Assignees

@liu-cong liu-cong

@ahg-g ahg-g

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. tide/merge-method-squash Denotes a PR that should be squashed by tide when it merges.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@coolkp @liu-cong @k8s-ci-robot @ahg-g @guydc