You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Remove EndpointPickerNotHealthy condition from InferencePool Status
Why is this needed:
InferencePool Status is typically managed by the gateway controller, which in the normal circumstances wouldn't monitor the epp health status, the proxy does, and so in most cases it will not be practical for the gateway controller to set it
Good point, so the tricky part is that not all gateway implementations have access to this signal to actually report it, and so it seems worse to define the condition, but it is mostly not implemented. I think it is reasonable to align with the capabilities of the the initial set of gateway controllers that aim to implement the API and expand later.
What would you like to be added:
Remove
EndpointPickerNotHealthy
condition from InferencePool StatusWhy is this needed:
InferencePool Status is typically managed by the gateway controller, which in the normal circumstances wouldn't monitor the epp health status, the proxy does, and so in most cases it will not be practical for the gateway controller to set it
/cc @danehans @robscott
The text was updated successfully, but these errors were encountered: