Skip to content

Commit 4ce70c4

Browse files
kfswainkaushikmitr
authored andcommitted
Adding a slim roadmap to README (kubernetes-sigs#400)
1 parent cf905a1 commit 4ce70c4

File tree

1 file changed

+10
-1
lines changed

1 file changed

+10
-1
lines changed

README.md

+10-1
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,16 @@ See our website at https://gateway-api-inference-extension.sigs.k8s.io/ for deta
2525

2626
## Roadmap
2727

28-
Coming soon!
28+
As Inference Gateway builds towards a GA release. We will continue to expand our capabilities, namely:
29+
1. Prefix-cache aware load balancing with interfaces for remote caches
30+
1. Recommended LoRA adapter pipeline for automated rollout
31+
1. Fairness and priority between workloads within the same criticality band
32+
1. HPA support for autoscaling on aggregate metrics derived from the load balancer
33+
1. Support for large multi-modal inputs and outputs
34+
1. Support for other GenAI model types (diffusion and other non-completion protocols)
35+
1. Heterogeneous accelerators - serve workloads on multiple types of accelerator using latency and request cost-aware load balancing
36+
1. Disaggregated serving support with independently scaling pools
37+
2938

3039
## End-to-End Tests
3140

0 commit comments

Comments
 (0)