README.md

Test

This directory contains tests and testing docs for Knative Serving:

• Unit tests currently reside in the codebase alongside the code they test
• End-to-end tests, of which there are two types:
  • Conformance tests in /test/conformance
  • Other end-to-end tests in /test/e2e
• Performance tests reside in /test/performance

The conformance tests are a subset of the end to end test with more strict requirements around what can be tested.

If you want to add more tests, see adding_tests.md.

Presubmit tests

presubmit-tests.sh is the entry point for both the end-to-end tests and the conformance tests

This script, and consequently, the e2e and conformance tests will be run before every code submission. You can run these tests manually with:

test/presubmit-tests.sh

Note that to run presubmit-tests.sh or e2e-tests.sh scripts, you'll need kubernetes kubetest installed:

go get -u k8s.io/test-infra/kubetest

Running unit tests

To run all unit tests:

go test ./...

By default go test will not run the e2e tests, which need -tags=e2e to be enabled.

Running end to end tests

To run the e2e tests and the conformance tests, you need to have a running environment that meets the e2e test environment requirements, and you need to specify the build tag e2e.

go test -v -tags=e2e -count=1 ./test/conformance
go test -v -tags=e2e -count=1 ./test/e2e

Running performance tests

To run the performance tests, you need to have a running environment that meets the test environment requirements, and you need to specify the build tag performance.

go test -v -tags=performance -count=1 ./test/performance

Running a single test case

To run one e2e test case, e.g. TestAutoscaleUpDownUp, use the -run flag with go test:

go test -v -tags=e2e -count=1 ./test/e2e -run ^TestAutoscaleUpDownUp$

Running tests in short mode

Running tests in short mode excludes some large-scale E2E tests and saves time/resources required for running the test suite. To run the tests in short mode, use the -short flag with go test

go test -v -tags=e2e -count=1 -short ./test/e2e

To get a better idea where the flag is used, search for testing.Short() throughout the test source code.

Environment requirements

These tests require:

1. A running Knative Serving cluster.

2. The knative-testing resources:

   ko apply -f test/config

3. The namespace serving-tests:

   kubectl create namespace serving-tests

4. A docker repo containing the test images

Common Flags

• By default the e2e tests against the current cluster in ~/.kube/config using the environment specified in your environment variables.
• Since these tests are fairly slow, running them with logging enabled is recommended (-v).
• Using --logverbose to see the verbose log output from test as well as from k8s libraries.
• Using -count=1 is the idiomatic way to disable test caching

You can use test flags to control the environment your tests run against, i.e. override your environment variables:

go test -v -tags=e2e -count=1 ./test/conformance --kubeconfig ~/special/kubeconfig --cluster myspecialcluster --dockerrepo myspecialdockerrepo
go test -v -tags=e2e -count=1 ./test/e2e --kubeconfig ~/special/kubeconfig --cluster myspecialcluster --dockerrepo myspecialdockerrepo

If you are running against an environment with no loadbalancer for the ingress, at the moment your only option is to use a domain which will resolve to the IP of the running node (see #609):

go test -v -tags=e2e -count=1 ./test/conformance --resolvabledomain
go test -v -tags=e2e -count=1 ./test/e2e --resolvabledomain

Test images

Building the test images

Note: this is only required when you run conformance/e2e tests locally with go test commands.

The upload-test-images.sh script can be used to build and push the test images used by the conformance and e2e tests. It requires:

• DOCKER_REPO_OVERRIDE to be set
• You to be authenticated with your DOCKER_REPO_OVERRIDE
• docker to be installed

To run the script for all end to end test images:

./test/upload-test-images.sh

A docker tag may be passed as an optional parameter. This can be useful on [Minikube] in tandem with the --tag flag:

eval $(minikube docker-env)
./test/upload-test-images.sh any-old-tag

Adding new test images

New test images should be placed in ./test/test_images.

Flags

These flags are useful for running against an existing cluster, making use of your existing environment setup.

Tests importing github.com/knative/serving/test recognize these flags:

• All flags added by knative/pkg/test
• --dockerrepo
• --tag
• --resolvabledomain

Overridding docker repo

The --dockerrepo argument lets you specify the docker repo from which images used by your tests should be pulled. This will default to the value of your DOCKER_REPO_OVERRIDE environment variable if not specified.

go test -v -tags=e2e -count=1 ./test/conformance --dockerrepo gcr.myhappyproject
go test -v -tags=e2e -count=1 ./test/e2e --dockerrepo gcr.myhappyproject

Using a docker tag

The default docker tag used for the test images is latest, which can be problematic on [Minikube]. To avoid having to configure a remote container registry to support the Always pull policy for latest tags, you can have the tests use a specific tag:

go test -v -tags=e2e -count=1 ./test/conformance --tag any-old-tag
go test -v -tags=e2e -count=1 ./test/e2e --tag any-old-tag

Of course, this implies that you tagged the images when you uploaded them.

Using a resolvable domain

If you set up your cluster using the getting started docs, Routes created in the test will use the domain example.com, unless the route has label app=prod in which case they will use the domain prod-domain.com. Since these domains will not be resolvable to deployments in your test cluster, in order to make a request against the endpoint, the test use the IP assigned to the service istio-ingressgateway in the namespace istio-system and spoof the Host in the header.

If you have configured your cluster to use a resolvable domain, you can use the --resolvabledomain flag to indicate that the test should make requests directly against Route.Status.Domain and does not need to spoof the Host.