Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Amend the endpoint picker protocol to support fallbacks and subsetting #445

Merged
merged 9 commits into from
Mar 13, 2025
Merged

Amend the endpoint picker protocol to support fallbacks and subsetting #445

merged 9 commits into from
Mar 13, 2025
+37 −7

Conversation

ahg-g
Copy link
Contributor

@ahg-g ahg-g commented Mar 3, 2025

Part of #415
Part of #414

@k8s-ci-robot k8s-ci-robot requested review from danehans and Jeffwan March 3, 2025 23:10
@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. approved Indicates a PR has been approved by an approver from all required OWNERS files. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels Mar 3, 2025
@netlify Netlify
Copy link

netlify bot commented Mar 3, 2025 •
edited
Loading

✅ Deploy Preview for gateway-api-inference-extension ready!

Name Link
🔨 Latest commit 41d5475
🔍 Latest deploy log https://app.netlify.com/sites/gateway-api-inference-extension/deploys/67d2e5260d2b240008448f59
😎 Deploy Preview https://deploy-preview-445--gateway-api-inference-extension.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

@ahg-g ahg-g force-pushed the fallbacks-protocol branch from 895d743 to 244e4d8 Compare March 3, 2025 23:11
@ahg-g ahg-g force-pushed the fallbacks-protocol branch from 244e4d8 to fb159a5 Compare March 3, 2025 23:12
LiorLieberman
LiorLieberman reviewed Mar 5, 2025
View reviewed changes
Copy link
Member

@LiorLieberman LiorLieberman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ahg-g !

ahg-g
ahg-g commented Mar 6, 2025
View reviewed changes
Copy link
Contributor Author

@ahg-g ahg-g left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks

@ahg-g ahg-g mentioned this pull request Mar 6, 2025
Closed
LiorLieberman
LiorLieberman approved these changes Mar 7, 2025
View reviewed changes
Copy link
Member

@LiorLieberman LiorLieberman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @ahg-g this looks good to me. Still need to experiment a little more with the fallback metadata to be sure, but it looks good overall.

AndresGuedez
AndresGuedez reviewed Mar 12, 2025
View reviewed changes
Copy link

@AndresGuedez AndresGuedez left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for adding this to the spec!

AndresGuedez
AndresGuedez approved these changes Mar 13, 2025
View reviewed changes
docs/proposals/004-endpoint-picker-protocol/README.md Outdated

## Matching An InferenceModel
The model name of a request MUST match the `Sepc.ModelName` parameter of one of the `InferenceModels` referencing the `InferencePool` managed by the EPP. Otherwise, the EPP MUST return a 404 status code.
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

typo: "Sepc"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

yanavlasov
yanavlasov approved these changes Mar 13, 2025
View reviewed changes
Copy link

@yanavlasov yanavlasov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM from Envoy side. But please clarify that status is for the immediate response.

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ahg-g, AndresGuedez, LiorLieberman, yanavlasov

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ahg-g
Copy link
Contributor Author

ahg-g commented Mar 13, 2025

LGTM from Envoy side. But please clarify that status is for the immediate response.

Thanks for verifying that there is path for Envoy to support this protocol

@ahg-g
Copy link
Contributor Author

ahg-g commented Mar 13, 2025

@kfswain can you pls another look, this should be ready

@danehans
Copy link
Contributor

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 13, 2025
@k8s-ci-robot k8s-ci-robot merged commit fb804b0 into kubernetes-sigs:main Mar 13, 2025
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@hzxuzhonghu hzxuzhonghu hzxuzhonghu left review comments

@kfswain kfswain kfswain left review comments

@yanavlasov yanavlasov yanavlasov approved these changes

@AndresGuedez AndresGuedez AndresGuedez approved these changes

@LiorLieberman LiorLieberman LiorLieberman approved these changes

@danehans danehans Awaiting requested review from danehans

@Jeffwan Jeffwan Awaiting requested review from Jeffwan

Assignees

@danehans danehans

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
8 participants
@ahg-g @k8s-ci-robot @danehans @yanavlasov @hzxuzhonghu @AndresGuedez @LiorLieberman @kfswain