|
1 |
| -# kube-router-e2e |
| 1 | +## End-To-End Testing Framework |
| 2 | + |
| 3 | +This framework uses [Ginkgo](https://onsi.github.io/ginkgo/#getting-ginkgo) -- a BDD-style Go testing framework built to help you efficiently write expressive and comprehensive tests. |
| 4 | + |
| 5 | +Assumming a cluster already exists, the framework allows you to run tests against it. |
| 6 | +- NOTE: As of Jan 10th, 2019, a cluster must first be setup beforehand; that is, we don't yet support automatic provisioning of a cluster, though we hope to have this feature in very shortly. :) |
| 7 | + |
| 8 | +## Tests |
| 9 | + |
| 10 | +Tests are categorized as follows. Additional categories may be added as desired. |
| 11 | + |
| 12 | +- Router: focus on pod-to-pod connectivity |
| 13 | +- Services: focus on kube-router's services functionality |
| 14 | +- Policy: focus on network policy enforcement |
| 15 | +- Runtime: tests that focus just on specific kube-router functionality and are not tied to kubernetes |
| 16 | + |
| 17 | +## Running Tests |
| 18 | + |
| 19 | +The following command will run all tests: |
| 20 | + |
| 21 | + |
| 22 | + $ ginkgo -- --kuberouter.SSHConfig='cd <path-to-Vagrantfile> && vagrant ssh-config' --kuberouter.testScope="ipv6-cluster" |
| 23 | + |
| 24 | + |
| 25 | +### Running Specific tests |
| 26 | + |
| 27 | +To run a subset of the tests, you can use ginkgo's [focus spec](https://onsi.github.io/ginkgo/#focused-specs). Here are some examples: |
| 28 | + |
| 29 | + $ ginkgo --focus="Router*" -- --kuberouter.SSHConfig='cd /home/awander/go/src/github.com/Arvinderpal/k8-ipv6 && vagrant ssh-config' --kuberouter.testScope="ipv6-cluster" |
| 30 | + |
| 31 | + $ ginkgo --focus="Service-Proxy*" -- --kuberouter.SSHConfig='cd /home/awander/go/src/github.com/Arvinderpal/k8-ipv6 && vagrant ssh-config' --kuberouter.testScope="ipv6-cluster" |
| 32 | + |
| 33 | +## Configuration Options |
| 34 | +``` |
| 35 | +--kuberouter.provisioner: specify a provisioner (e.g. vagrant) |
| 36 | +--kuberouter.provision: provision a cluster before running the tests |
| 37 | +--kuberouter.skipLogs: skip gathering logs if a test fails |
| 38 | +--kuberouter.SSHConfig: specify a custom command to fetch SSH configuration (eg: 'vagrant ssh-config') |
| 39 | +--kuberouter.showCommands: output which commands are ran to stdout |
| 40 | +``` |
| 41 | +## Logs/Results |
| 42 | + |
| 43 | +For each test, we log: |
| 44 | +1. List commands that were issued (cmds.log) |
| 45 | +2. Output of kube-router instances on the 2 nodes during the the time the specific test was running (kuberouter-test-k8s(1/2).s) |
| 46 | +3. Output of each command (test-out.log). |
| 47 | +4. If the test fails, we also save the entire kube-router log file (kube-router-complete-k8s1.s). |
| 48 | +These files can be found in the directory `test_results` under the test name. For example, |
| 49 | + |
| 50 | +``` |
| 51 | +~/go/src/github.com/cloudnativelabs/kube-router-e2e/test_results/Service-Proxy_Basic_Connectivity_to_Nodeport_Service_with_many_replicas $ ll |
| 52 | +total 12140 |
| 53 | +drwxrwxr-x 2 awander awander 4096 Jan 10 12:34 ./ |
| 54 | +drwxrwxr-x 4 awander awander 4096 Jan 10 12:29 ../ |
| 55 | +-rw-rw-r-- 1 awander awander 5417 Jan 10 12:34 cmds.log |
| 56 | +-rw-rw-r-- 1 awander awander 199945 Jan 10 12:29 kuberouter-test.log-k8s1.s |
| 57 | +-rw-rw-r-- 1 awander awander 16566 Jan 10 12:34 test-output.log |
| 58 | +``` |
| 59 | + |
| 60 | +Here is an example cmds.log file: |
| 61 | +``` |
| 62 | +curl -g -6 http://[::1]:20244/healthz |
| 63 | +gobgp neighbor |
| 64 | +kubectl apply -f /home/vagrant/go/src/github.com/cloudnativelabs/kube-router-e2e/manifests/busybox-1.yaml |
| 65 | +kubectl -n default get pods -l app=busybox-2 -o json |
| 66 | +sudo journalctl -au kube-router --since '60 seconds ago' |
| 67 | +kubectl delete -f /home/vagrant/go/src/github.com/cloudnativelabs/kube-router-e2e/manifests/busybox-2.yaml --grace-period=0 --force |
| 68 | +``` |
| 69 | + |
| 70 | +And here is a snippet of he kuberouter-test.log-k8s1.s file: |
| 71 | +``` |
| 72 | +-- Logs begin at Tue 2018-08-28 17:10:09 UTC, end at Tue 2019-01-15 23:47:44 UTC. -- |
| 73 | +Jan 15 23:46:46 k8s1 kube-router[4871]: I0115 23:46:46.615714 4871 network_routes_controller.go:252] Syncing ipsets |
| 74 | +Jan 15 23:46:46 k8s1 kube-router[4871]: I0115 23:46:46.661630 4871 network_routes_controller.go:265] Performing periodic sync of service VIP routes |
| 75 | +Jan 15 23:46:46 k8s1 kube-router[4871]: I0115 23:46:46.661699 4871 ecmp_vip.go:24] Advertising route: '172.20.0.10/32 via 192.168.33.8' to peers |
| 76 | +Jan 15 23:46:46 k8s1 kube-router[4871]: I0115 23:46:46.661717 4871 network_routes_controller.go:269] Performing periodic sync of pod CIDR routes |
| 77 | +``` |
| 78 | + |
| 79 | + |
| 80 | +## Acknowledgement |
| 81 | + |
| 82 | +This framework is based on the e2e testing framework of the [cilium project](https://github.com/cilium/cilium/tree/master/test). Many thanks to them for their excellent work! |
0 commit comments