Polish the epp README.md file

Author: ahg-g

pkg/epp/README.md

@@ -0,0 +1,16 @@
+# The EndPoint Picker (EPP)
+This pkg hosts the reference implementation of the Endpoint Picker, or EPP. It implements the [extension protocol](../../docs/proposals/003-endpoint-picker-protocol) which defines how a proxy can callout to an extension to request for endpoint hints. As it is implemented now, an EPP instance handles a single `InferencePool` (and so for each `InferencePool`, one must create a dedicated EPP deployment).
+
+The EPP does the following:
+
+- Picks the endpoint that the LB should route the request to. It picks from the set of ready Pods selected by the assigned `InferencePool`. It processes requests only when the ModelName matches an `InferenceModel` referencing the assigned `InferencePool`. Unmatched requests result in an error sent to the proxy.". The algorithm for picking the endpoint is described below.
+
+- Traffic splitting between adapter versions. `InferenceModel` allows for defining traffic splitting between adapters in the same InferencePool to allow for rollouts of new adapter versions.
+
+- Emitting metrics for observability: EPP will report InferenceModel level metrics broken down by target model as well, the details are at GKE Inference Gateway Observability Proposal. 
+
+## The scheduling algorithm 
+The scheduling package implements request scheduling algorithms for load balancing requests across backend pods in an inference gateway. The scheduler ensures efficient resource utilization while maintaining low latency and prioritizing critical requests. It applies a series of filters based on metrics and heuristics to select the best pod for a given request. The following flow chart summarizes the current scheduling algorithm
+
+# Flowchart
+<img src="../docs/schedular-flowchart.png" alt="Scheduling Algorithm" width="400" />