Skip to content

Commit c204cf7

Browse files
committed
address comments
Signed-off-by: wangbaiping(wbpcode) <[email protected]>
1 parent f7d96a8 commit c204cf7

File tree

1 file changed

+3
-15
lines changed
  • docs/proposals/004-endpoint-picker-protocol

1 file changed

+3
-15
lines changed

docs/proposals/004-endpoint-picker-protocol/README.md

Lines changed: 3 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -25,9 +25,9 @@ If the key `x-gateway-destination-endpoint-subset` is set, the EPP MUST only sel
2525
If the key `x-gateway-destination-endpoint-subset` is not set, then the EPP MUST select from the set defined by the `InferencePool` selector.
2626

2727
## Destination Endpoint
28-
For each HTTP request, the EPP MUST communicate to the proxy the picked model server endpoint via:
28+
For each HTTP request, the EPP MUST communicate to the proxy one or more selected model server endpoints via:
2929

30-
1. Setting the `x-gateway-destination-endpoint` HTTP header to the selected endpoints.
30+
1. Setting the `x-gateway-destination-endpoint` HTTP header to one or more selected endpoints.
3131

3232
2. Set an unstructured entry in the [dynamic_metadata](https://github.com/envoyproxy/go-control-plane/blob/c19bf63a811c90bf9e02f8e0dc1dcef94931ebb4/envoy/service/ext_proc/v3/external_processor.pb.go#L320) field of the ext-proc response. The metadata entry for the picked endpoints MUST be wrapped with an outer key (which represents the metadata namespace) with a default of `envoy.lb`.
3333

@@ -48,19 +48,7 @@ dynamicMetadata: {
4848
}
4949
```
5050

51-
The value of the header or metadata entry MUST contains at least one endpoint in `<ip:port>` format or multiple endpoints in `<ip:port>,<ip:port>,...` format. Multiple endpoints are separated by commas. The first valid endpoint in the value will be used. And if retrying is happening, the proxy will try the endpoints after the previously selected endpoint in order.
52-
53-
Optionally, The EPP also CAN set additional endpoints by the key `x-gateway-destination-endpoint-fallback` in the same metadata namespace as one used for `x-gateway-destination-endpoint` as follows:
54-
55-
```go
56-
dynamicMetadata: {
57-
"envoy.lb" {
58-
"x-gateway-destination-endpoint-fallback": <ip:port>
59-
}
60-
}
61-
```
62-
63-
The endpoints specified in `x-gateway-destination-endpoint-fallback` MAY be tried after the endpoints specified in `x-gateway-destination-endpoint` if all the endpoints specified in `x-gateway-destination-endpoint` are unavailable.
51+
The value of the header or metadata entry MUST contain at least one endpoint in `<ip:port>` format or multiple endpoints in `<ip:port>,<ip:port>,...` format. Multiple endpoints are separated by commas. The first valid endpoint in the list will be used. And if retrying is happening, the proxy will try the endpoints after the previously selected endpoint in order.
6452

6553
Constraints:
6654
- If the EPP did not communicate the server endpoint via these two methods, it MUST return an error as follows:

0 commit comments

Comments
 (0)