Provide Flexability in InferencePool Assignment

[danehans]

Currently, EPP will only reconcile an InferencePool that matches its configured poolName CLI flag. A more flexible approach should be considered since changing InferencePools requires a restart. One approach to consider is using labels, where EPP can use a predicate to filter reconciling all InferencePools in its namespace to only one that matches its configured poolLabel matcher. EPP can follow Gateway API guidelines for conflict resolution.

[danehans]

@ahg-g @kfswain @robscott PTAL and provide your thoughts when you have a moment.

[ahg-g]

I am supportive, would you like to make a concrete proposal? We need to resolve a few things:

1. If we are going with a label, define a label key
2. How to ensure that this aligns with the configuration api that defines a service reference? Should the EPP look at that instead? or spec that the EPP service reference and the label must be aligned, and if not the behavior is unknown?
3. Since our current implementation of EPP assumes a single inferencePool, we need to also define a predictable behavior as to what exact pool the EPP will serve until we decide whether or not we want EPP to support multiple pools.