generated from kubernetes/kubernetes-template-project
-
Notifications
You must be signed in to change notification settings - Fork 90
Adding getting started instructions for GKE, Istio, and Kgateway #577
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 31 commits
Commits
Show all changes
52 commits
Select commit
Hold shift + click to select a range
6bc07c6
Create resources.yaml for kgateway
nicolexin 63d7c40
Update getting started guide for KGateway
nicolexin 048189a
Replace Envoy Gateway user guide with GKE user guide
nicolexin a679070
Create resources.yaml for GKE Gateway
nicolexin a627ea7
Delete config/manifests/gateway/enable_patch_policy.yaml
nicolexin 7b490de
Delete config/manifests/gateway/gateway.yaml
nicolexin 9c8d00d
Delete config/manifests/gateway/patch_policy.yaml
nicolexin 0519935
Delete config/manifests/gateway/traffic_policy.yaml
nicolexin 3e7e74e
Add http2 appProtocol to EPP service
nicolexin a140a3e
Add user guide for Istio
nicolexin 8a878f8
Create resources.yaml for Istio
nicolexin f0b59e4
Fix GKE gateway name to match the user guide
nicolexin c06cffd
Fix cleanup instructions to refer up-to-date YAMLs
nicolexin 21100f9
Allow Istio gateway to use HTTPRoute from all namespaces
nicolexin d8d4666
Update Kgateway port number to 80
nicolexin ee7fa97
Update gateway port to 80
nicolexin 59cbe2e
Remove the sectionName from Kgateway HTTPRoute
nicolexin afc64dc
Create common httproute YAML
nicolexin 8d235f6
Create healthcheck.yaml for GKE gateway
nicolexin 52318b3
Separate gateway.yaml for GKE gateway
nicolexin 9343660
Separate gateway.yaml for Istio
nicolexin 8ef12a8
Separate gateway.yaml for Kgateway
nicolexin 557c44f
Update the user guide to use shared HTTPRoute YAML
nicolexin 6d48b5b
Add EPP DestinationRule for Istio
nicolexin e512145
Add instructions for bypassing TLS verification for Istio
nicolexin e82e074
Update CRDs to the latest v0.2.0 release
nicolexin ff8b2a1
Update gateway to use the v1 API
nicolexin f6f9538
Remove weight from HTTPRoute
nicolexin efb8c35
Update gateway.yaml
nicolexin 5a2677e
Remove allowedRoutes from Istio gateway
nicolexin ce19438
Remove allowedRoutes from Kgateway
nicolexin b63263d
Merge branch 'kubernetes-sigs:main' into userguide
nicolexin d493258
Update latest instructions for installing Istio and addressing some c…
nicolexin 9cb2575
Fix indentation for installing CRDs
nicolexin 2574453
Merge remote-tracking branch 'upstream/main' into userguide
nicolexin 2f9baea
Merge branch 'kubernetes-sigs:main' into userguide
nicolexin 35a835f
Addressing code review comments
nicolexin 0a24389
Fix indentation
nicolexin c1b563b
Update Istio installation instructions
nicolexin 6d3642a
Fix indentation
nicolexin 6a9f91a
Fix indentation
nicolexin b6d4c7a
Add more spacing to the CPU based model instructions
nicolexin e9f2298
Removing comments from kgateway
nicolexin 484f19f
Add clarification on the EPP secureServing default value.
nicolexin d71f29c
Add instructions for configuring timeout
nicolexin 41fc083
Create httproute-with-timeout.yaml
nicolexin d5fd70f
Create gcp-backend-policy.yaml
nicolexin d0ddd16
Add cleanup for GCPBackendPolicy
nicolexin e1c0b1d
Remove namespace from destination-rule.yaml
nicolexin e4471ec
Rename inferencepool.yaml to inferencepool-resources.yaml
nicolexin 365d847
Rename inferencepool.yaml to inferencepool-resources.yaml
nicolexin c82487d
Rename inferencepool.yaml to inferencepool-resources.yaml
nicolexin File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| kind: Gateway | ||
| apiVersion: gateway.networking.k8s.io/v1 | ||
| metadata: | ||
| name: inference-gateway | ||
| spec: | ||
| gatewayClassName: gke-l7-regional-external-managed | ||
| listeners: | ||
| - name: http | ||
| port: 80 | ||
| protocol: HTTP |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| kind: HealthCheckPolicy | ||
| apiVersion: networking.gke.io/v1 | ||
| metadata: | ||
| name: health-check-policy | ||
| namespace: default | ||
| spec: | ||
| targetRef: | ||
| group: "inference.networking.x-k8s.io" | ||
| kind: InferencePool | ||
| name: vllm-llama2-7b | ||
| default: | ||
| config: | ||
| type: HTTP | ||
| httpHealthCheck: | ||
| requestPath: /health | ||
| port: 8000 | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| apiVersion: gateway.networking.k8s.io/v1 | ||
| kind: HTTPRoute | ||
| metadata: | ||
| name: llm-route | ||
| spec: | ||
| parentRefs: | ||
| - group: gateway.networking.k8s.io | ||
| kind: Gateway | ||
| name: inference-gateway | ||
| rules: | ||
| - backendRefs: | ||
| - group: inference.networking.x-k8s.io | ||
| kind: InferencePool | ||
| name: vllm-llama2-7b | ||
| matches: | ||
| - path: | ||
| type: PathPrefix | ||
| value: / |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| apiVersion: networking.istio.io/v1 | ||
| kind: DestinationRule | ||
| metadata: | ||
| name: epp-insecure-tls | ||
| spec: | ||
| host: vllm-llama2-7b-epp.default.svc.cluster.local | ||
nicolexin marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| trafficPolicy: | ||
| tls: | ||
| mode: SIMPLE | ||
| insecureSkipVerify: true | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| apiVersion: gateway.networking.k8s.io/v1 | ||
| kind: Gateway | ||
| metadata: | ||
| name: inference-gateway | ||
| spec: | ||
| gatewayClassName: istio | ||
| listeners: | ||
| - name: http | ||
| port: 80 | ||
| protocol: HTTP |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| # Requires Kgateway 2.0.0 or greater. | ||
| --- | ||
nicolexin marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| apiVersion: gateway.networking.k8s.io/v1 | ||
| kind: Gateway | ||
| metadata: | ||
| name: inference-gateway | ||
| spec: | ||
| gatewayClassName: kgateway | ||
| listeners: | ||
| - name: http | ||
| port: 80 | ||
| protocol: HTTP | ||
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
nicolexin marked this conversation as resolved.
Show resolved
Hide resolved
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.