Adds Initial e2e Tests and Tooling #217

danehans · 2025-01-23T00:47:12Z

Adds initial e2e tests and tooling. The e2e tests can be run by using the test-e2e target. The "HF_TOKEN" environment variable must be set to your Hugging Face access token (with access to Llama2 model) before running the tests.

api/v1alpha1/inferencemodel_types.go and api/v1alpha1/inferencepool_types.go: Adds constants to define resource and kind.
Bumps go deps.
pkg/crd/install.go: Adds public function for installing CRDs. Used by e2e suite to setup test infra.
pkg/crd/install_test.go and pkg/crd/mocks/mock_client.go: Adds unit tests and mock client for InstallCRDs function.
test/consts/consts.go: Defines constants used by tests.
test/e2e/e2e_suite_test.go: Creates initial test suite, test infra, utility functions, etc.
test/e2e/e2e_test.go: Adds initial e2e test by creating an InferenceModel and ensuring that requests are properly load balanced across the target models.
test/utils/resources.go: Utility functions for managing Kubernetes resources used for e2e tests.
test/utils/utils.go: Provides utility functions for working with test resources.
test/utils/wrappers.go: Provides wrappers for working with test resources.

Fixes #77

netlify · 2025-01-23T00:47:30Z

✅ Deploy Preview for gateway-api-inference-extension ready!

Name	Link
🔨 Latest commit	`2319c1f`
🔍 Latest deploy log	https://app.netlify.com/sites/gateway-api-inference-extension/deploys/67992fcb606c100008ac33a9
😎 Deploy Preview	https://deploy-preview-217--gateway-api-inference-extension.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

ahg-g

This is great!

One concern I have is that we are coding a lot of things that could have been done easier with kubectl and a shell script, specifically: deploying the crds, the envoy proxy and the epp. What I am thinking is that why not have a script that deploys everything as described in our guide, then in go we verify that everything is up and running (via a BeforeSuite) and runs the test client.

There are a couple of advantages to this approach:

significantly simplify the golang code
reduce duplication since we will be reusing the manifests we have for the user guide
practically this e2e test is a test for the guide as well!

what do you think?

api/v1alpha1/inferencemodel_types.go

api/v1alpha1/inferencepool_types.go

pkg/crd/install.go

test/e2e/e2e_test.go

k8s-ci-robot · 2025-01-27T21:26:43Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: danehans

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [danehans]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

danehans · 2025-01-27T22:10:46Z

pkg/README.md


-   ```sh
-   kubectl apply -f config/crd/bases
+   Replace `$HF_TOKEN` in `./manifests/vllm/deployment.yaml` with your Hugging Face secret and then deploy the sample vLLM deployment.


This step was updated since the e2e test suite now deploys vLLM test infra resources from this manifest by:

Reading the manifest file into memory

Reading the HF_TOKEN env var

Replacing $HF_TOKEN with the HF_TOKEN env var in the in-memory manifest

Deploying the resources

danehans · 2025-01-27T22:13:38Z

pkg/README.md

   ```bash
-   kubectl apply -f ./manifests/inferencepool-with-model.yaml
+   kubectl apply -f ./manifests/inferencemodel.yaml
   ```


Note that the InferencePool CR is now part of the ext-proxy deployment. The InferenceModel had to be split from the InferencePool since e2e tests will create InferenceModels for each test case. The InferenceModel could have its own manifest but IMHO it makes sense to bundle with the ext-proc since the ext-proc is specifically configured for this InferencePool.

danehans · 2025-01-27T22:14:22Z

pkg/README.md


-1. **Deploy Ext-Proc**
+1. **Deploy the Inference Extension and InferencePool**


As I mentioned above, the InferencePool is now bundled with the ext-proc since the ext-proc is specifically configured for this pool.

danehans · 2025-01-27T22:15:11Z

pkg/manifests/vllm/deployment.yaml

+  labels:
+    app: vllm
+stringData:
+  token: $HF_TOKEN


e2e will replace this variable and the POC instructions have been updated accordingly.

I think we should have this Secret resource in a separate yaml so that the guide doesn't require the user to clone the repo to test it out.

danehans · 2025-01-27T22:29:07Z

@ahg-g I updated the e2e test suite to use a "golden manifest" approach. This approach provides the following benefits:

Single Flow & Simplified Maintenance: A single test suite that runs both the “deploy” and “test” steps, making it simpler to maintain and debug.
Deterministic, Self-Contained Tests: The test harness ensures that each test run has the same environment—no reliance on manual or separate script-driven steps.
Better Control of Race Conditions & Ordering: Runs custom wait logic (or test verifications) between each resource’s creation. Scripts typically proceed line by line without advanced logic unless heavily scripted.
Native Go Logging, Assertions, & Error Handling: Failures integrate seamlessly with the test harness (e.g., Ginkgo, testing.T), producing richer debug output.

ahg-g

This is awesome, can we add a readme how to run this test manually pls?

pkg/README.md

ahg-g · 2025-01-28T17:41:51Z

pkg/manifests/vllm/deployment.yaml

+  labels:
+    app: vllm
+stringData:
+  token: $HF_TOKEN


I think we should have this Secret resource in a separate yaml so that the guide doesn't require the user to clone the repo to test it out.

Signed-off-by: Daneyon Hansen <[email protected]>

ahg-g · 2025-01-28T20:18:50Z

Amazing!

/lgtm

k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. labels Jan 23, 2025

danehans requested review from robscott and kfswain January 23, 2025 00:47

danehans force-pushed the issue_77 branch from 5a01240 to 909925f Compare January 23, 2025 00:53

danehans mentioned this pull request Jan 23, 2025

Separate and organize endpoint selector manifests #202

Closed

danehans force-pushed the issue_77 branch from 909925f to 04dae58 Compare January 23, 2025 17:38

ahg-g reviewed Jan 25, 2025

View reviewed changes

api/v1alpha1/inferencemodel_types.go Outdated Show resolved Hide resolved

api/v1alpha1/inferencepool_types.go Outdated Show resolved Hide resolved

pkg/crd/install.go Outdated Show resolved Hide resolved

test/e2e/e2e_test.go Outdated Show resolved Hide resolved

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 27, 2025

danehans force-pushed the issue_77 branch from 04dae58 to 953235d Compare January 27, 2025 21:26

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 27, 2025

danehans force-pushed the issue_77 branch from 953235d to 7c4e3c5 Compare January 27, 2025 22:00

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 27, 2025

danehans force-pushed the issue_77 branch from 7c4e3c5 to 6b2fd93 Compare January 27, 2025 22:03

danehans commented Jan 27, 2025

View reviewed changes

danehans force-pushed the issue_77 branch from 6b2fd93 to 878ab9a Compare January 27, 2025 22:23

ahg-g reviewed Jan 28, 2025

View reviewed changes

k8s-ci-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 28, 2025

danehans force-pushed the issue_77 branch from 878ab9a to 203ce4b Compare January 28, 2025 16:27

k8s-ci-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 28, 2025

danehans requested a review from ahg-g January 28, 2025 17:47

ahg-g reviewed Jan 28, 2025

View reviewed changes

danehans mentioned this pull request Jan 28, 2025

Refactor ext-proc Main with Server Package Add Hermetic Test with k8s API Client for EPP #222

Merged

Adds initial e2e tests and tooling

0c1c780

Signed-off-by: Daneyon Hansen <[email protected]>

danehans added 3 commits January 28, 2025 19:07

Refactors e2e for manifest approach

4c50a37

Signed-off-by: Daneyon Hansen <[email protected]>

Adds e2e test readme

4818300

Signed-off-by: Daneyon Hansen <[email protected]>

Uses a separate model server secret for e2e

2319c1f

Signed-off-by: Daneyon Hansen <[email protected]>

danehans force-pushed the issue_77 branch from 464d8a3 to 2319c1f Compare January 28, 2025 19:28

k8s-ci-robot assigned ahg-g Jan 28, 2025

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jan 28, 2025

k8s-ci-robot merged commit 16b95a2 into kubernetes-sigs:main Jan 28, 2025
8 checks passed

danehans deleted the issue_77 branch January 28, 2025 20:19

danehans mentioned this pull request Jan 28, 2025

e2e Tests Fail to Validate e2e Functionality #239

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds Initial e2e Tests and Tooling #217

Adds Initial e2e Tests and Tooling #217

danehans commented Jan 23, 2025

netlify bot commented Jan 23, 2025 •

edited

Loading

ahg-g left a comment

k8s-ci-robot commented Jan 27, 2025

danehans Jan 27, 2025

danehans Jan 27, 2025

danehans Jan 27, 2025

danehans Jan 27, 2025

ahg-g Jan 28, 2025

danehans commented Jan 27, 2025

ahg-g left a comment

ahg-g Jan 28, 2025

ahg-g commented Jan 28, 2025


		1. Deploy Ext-Proc
		1. Deploy the Inference Extension and InferencePool

Adds Initial e2e Tests and Tooling #217

Adds Initial e2e Tests and Tooling #217

Conversation

danehans commented Jan 23, 2025

netlify bot commented Jan 23, 2025 • edited Loading

✅ Deploy Preview for gateway-api-inference-extension ready!

ahg-g left a comment

Choose a reason for hiding this comment

k8s-ci-robot commented Jan 27, 2025

danehans Jan 27, 2025

Choose a reason for hiding this comment

danehans Jan 27, 2025

Choose a reason for hiding this comment

danehans Jan 27, 2025

Choose a reason for hiding this comment

danehans Jan 27, 2025

Choose a reason for hiding this comment

ahg-g Jan 28, 2025

Choose a reason for hiding this comment

danehans commented Jan 27, 2025

ahg-g left a comment

Choose a reason for hiding this comment

ahg-g Jan 28, 2025

Choose a reason for hiding this comment

ahg-g commented Jan 28, 2025

netlify bot commented Jan 23, 2025 •

edited

Loading