Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove EndpointPickerNotHealthy condition from InferencePool Status #385

Closed
Tracked by #362
ahg-g opened this issue Feb 21, 2025 · 2 comments · Fixed by #421
Closed
Tracked by #362

Remove EndpointPickerNotHealthy condition from InferencePool Status #385

ahg-g opened this issue Feb 21, 2025 · 2 comments · Fixed by #421

Comments

@ahg-g
Copy link
Contributor

ahg-g commented Feb 21, 2025

What would you like to be added:

Remove EndpointPickerNotHealthy condition from InferencePool Status

Why is this needed:

InferencePool Status is typically managed by the gateway controller, which in the normal circumstances wouldn't monitor the epp health status, the proxy does, and so in most cases it will not be practical for the gateway controller to set it

/cc @danehans @robscott

@kfswain
Copy link
Collaborator

kfswain commented Feb 21, 2025

How will we signal poor Pool health?

@ahg-g
Copy link
Contributor Author

ahg-g commented Feb 21, 2025

Good point, so the tricky part is that not all gateway implementations have access to this signal to actually report it, and so it seems worse to define the condition, but it is mostly not implemented. I think it is reasonable to align with the capabilities of the the initial set of gateway controllers that aim to implement the API and expand later.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants