Skip to content
This repository was archived by the owner on May 12, 2021. It is now read-only.

Commit 3655b4c

Browse files
author
Eric Ernst
authored
Merge pull request #445 from grahamwhaley/20190424_minikube_docs
install: Add minikube instructions
2 parents dfda61c + 8e29464 commit 3655b4c

File tree

2 files changed

+233
-1
lines changed

2 files changed

+233
-1
lines changed

‎install/README.md

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -96,8 +96,9 @@ Manual installation instructions are available for [these distributions](#suppor
9696

9797
## Installing on a Cloud Service Platform
9898
* [Amazon Web Services (AWS)](aws-installation-guide.md)
99-
* [Microsoft Azure](azure-installation-guide.md)
10099
* [Google Compute Engine (GCE)](gce-installation-guide.md)
100+
* [Microsoft Azure](azure-installation-guide.md)
101+
* [Minikube](minikube-installation-guide.md)
101102
* [VEXXHOST OpenStack Cloud](vexxhost-installation-guide.md)
102103

103104
## Further information

‎install/minikube-installation-guide.md

Lines changed: 231 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,231 @@
1+
# Installing Kata Containers in Minikube
2+
3+
* [Installing Kata Containers in Minikube](#installing-kata-containers-in-minikube)
4+
* [Introduction](#introduction)
5+
* [Prerequisites](#prerequisites)
6+
* [Setting up Minikube](#setting-up-minikube)
7+
* [Checking for nested virtualization](#checking-for-nested-virtualization)
8+
* [Check Minikube is running](#check-minikube-is-running)
9+
* [Installing Kata Containers](#installing-kata-containers)
10+
* [Enabling Kata Containers](#enabling-kata-containers)
11+
* [Register the runtime](#register-the-runtime)
12+
* [Testing Kata Containers](#testing-kata-containers)
13+
* [Wrapping up](#wrapping-up)
14+
15+
## Introduction
16+
17+
[Minikube](https://kubernetes.io/docs/setup/minikube/) is an easy way to try out a Kubernetes (k8s)
18+
cluster locally. It creates a single node Kubernetes stack in a local VM.
19+
20+
[Kata Containers](https://github.com/kata-containers) can be installed into a Minikube cluster using
21+
[`kata-deploy`](https://github.com/kata-containers/packaging/tree/master/kata-deploy).
22+
23+
This document details the pre-requisites, installation steps, and how to check
24+
the installation has been successful.
25+
26+
## Prerequisites
27+
28+
This installation guide has only been verified under a Minikube Linux installation, using the
29+
[`kvm2`](https://github.com/kubernetes/minikube/blob/master/docs/drivers.md#kvm2-driver) driver.
30+
31+
> **Notes:**
32+
> - This installation guide may not work for macOS installations of Minikube, due to the lack of
33+
nested virtualization support on that platform.
34+
> - This installation guide has not been tested on a Windows installation.
35+
36+
Before commencing installation, it is strongly recommended you read the
37+
[Minikube installation guide](https://kubernetes.io/docs/tasks/tools/install-minikube/).
38+
39+
## Checking for nested virtualization
40+
41+
For Kata Containers to work under a Minikube VM, your host system must support
42+
nested virtualization. If you are using a Linux system utilizing Intel VT-x
43+
and the `kvm_intel` driver, you can perform the following check:
44+
45+
```sh
46+
$ cat /sys/module/kvm_intel/parameters/nested
47+
```
48+
49+
If your system does not report `Y` from the `nested` parameter, then details on how
50+
to enable nested virtualization can be found on the
51+
[KVM Nested Guests page](https://www.linux-kvm.org/page/Nested_Guests)
52+
53+
Alternatively, and for other architectures, the Kata Containers built in
54+
[`kata-check`](https://github.com/kata-containers/runtime#hardware-requirements)
55+
command can be used *inside Minikube* once Kata has been installed, to check for compatibility.
56+
57+
## Setting up Minikube
58+
59+
To enable Kata Containers under Minikube, you need to add a few configuration options to the
60+
default Minikube setup. You can easily accomplish this as Minikube supports them on the setup commandline.
61+
62+
Here are the features, and why you need them:
63+
64+
| what | why |
65+
| ---- | --- |
66+
| `--bootstrapper=kubeadm` | As recommended for [minikube CRI-o](https://kubernetes.io/docs/setup/minikube/#cri-o) |
67+
| `--container-runtime=cri-o` | Using CRI-O for Kata |
68+
| `--enable-default-cni` | As recommended for [minikube CRI-o](https://kubernetes.io/docs/setup/minikube/#cri-o) |
69+
| `--memory 6144` | Allocate sufficient memory, as Kata Containers default to 1 or 2Gb |
70+
| `--network-plugin=cni` | As recommended for [minikube CRI-o](https://kubernetes.io/docs/setup/minikube/#cri-o) |
71+
| `--vm-driver kvm2` | The host VM driver |
72+
73+
> **Notes:**
74+
> - Adjust the `--memory 6144` line to suit your environment and requirements. Kata Containers default to
75+
> requesting 2048MB per container. We recommended you supply more than that to the Minikube node.
76+
> - This example deploys Kata Containers using CRI-O. The same procedure should also work
77+
> for a `containerd` based Kubernetes installation, but we do not cover that in this document.
78+
>
79+
> - Prior to Minikube/Kubernetes v1.14, the beta `RuntimeClass` feature also needed enabling with
80+
> the following.
81+
>
82+
> | what | why |
83+
> | ---- | --- |
84+
> | `--feature-gates=RuntimeClass=true` | Kata needs to use the `RuntimeClass` Kubernetes feature |
85+
86+
The full command is therefore:
87+
88+
```sh
89+
$ minikube start --vm-driver kvm2 --memory 6144 --network-plugin=cni --enable-default-cni --container-runtime=cri-o --bootstrapper=kubeadm
90+
```
91+
92+
> **Note:** For Kata Containers later than v1.6.1, the now default `tcfilter` networking of Kata Containers
93+
> does not work for Minikube versions less than v1.1.1. Please ensure you use Minikube version v1.1.1
94+
> or above.
95+
96+
## Check Minikube is running
97+
98+
Before you install Kata Containers, check that your Minikube is operating. On your guest:
99+
100+
101+
```sh
102+
$ kubectl get nodes
103+
```
104+
105+
You should see your `master` node listed as being `Ready`.
106+
107+
Check you have virtualization enabled inside your Minikube. The following should return
108+
a number larger than `0` if you have either of the `vmx` or `svm` nested virtualization features
109+
available:
110+
111+
```sh
112+
$ minikube ssh "egrep -c 'vmx|svm' /proc/cpuinfo"
113+
```
114+
115+
## Installing Kata Containers
116+
117+
You can now install the Kata Containers runtime components. You will need a local copy of some Kata
118+
Containers components to help with this, and then use `kubectl` on the host (that Minikube has already
119+
configured for you) to deploy them:
120+
121+
```sh
122+
$ git clone https://github.com/kata-containers/packaging.git
123+
$ cd packaging/kata-deploy
124+
$ kubectl apply -f kata-rbac.yaml
125+
$ kubectl apply -f kata-deploy.yaml
126+
```
127+
128+
This installs the Kata Containers components into `/opt/kata` inside the Minikube node. It can take
129+
a few minutes for the operation to complete. You can check the installation has worked by checking
130+
the status of the `kata-deploy` pod, which will be executing
131+
[this script](https://github.com/kata-containers/packaging/blob/master/kata-deploy/scripts/kata-deploy.sh),
132+
and will be executing a `sleep infinity` once it has successfully completed its work.
133+
You can accomplish this by running the following:
134+
135+
```sh
136+
$ podname=$(kubectl -n kube-system get pods -o=name | fgrep kata-deploy | sed 's?pod/??')
137+
$ kubectl -n kube-system exec ${podname} -- ps -ef | fgrep infinity
138+
```
139+
140+
> *NOTE:* This check only works for single node clusters, which is the default for Minikube.
141+
> For multi-node clusters, the check would need to be adapted to check `kata-deploy` had
142+
> completed on all nodes.
143+
144+
## Enabling Kata Containers
145+
146+
> **Note:** Only Minikube/Kubernetes versions <= 1.13 require this step. Since version
147+
> v1.14, the `RuntimeClass` is enabled by default. Performing this step on Kubernetes > v1.14 is
148+
> however benign.
149+
150+
Now you have installed the Kata Containers components in the Minikube node. Next, you need to configure
151+
Kubernetes `RuntimeClass` to know when to use Kata Containers to run a pod.
152+
153+
```sh
154+
$ kubectl apply -f https://raw.githubusercontent.com/kubernetes/node-api/master/manifests/runtimeclass_crd.yaml > runtimeclass_crd.yaml
155+
```
156+
157+
### Register the runtime
158+
159+
Now register the `kata qemu` runtime with that class. This should result in no errors:
160+
161+
```sh
162+
$ cd packaging/kata-deploy/k8s-1.14
163+
$ kubectl apply -f kata-qemu-runtimeClass.yaml
164+
```
165+
166+
The Kata Containers installation process should be complete and enabled in the Minikube cluster.
167+
168+
## Testing Kata Containers
169+
170+
Launch a container that has been defined to run on Kata Containers. The enabling is configured by
171+
the following lines in the YAML file. See the Kubernetes
172+
[Runtime Class Documentation](https://kubernetes.io/docs/concepts/containers/runtime-class/#usage)
173+
for more details.
174+
175+
```yaml
176+
spec:
177+
runtimeClassName: kata-qemu
178+
```
179+
180+
Perform the following action to launch a Kata Containers based Apache PHP pod:
181+
182+
```sh
183+
$ cd packaging/kata-deploy/examples
184+
$ kubectl apply -f test-deploy-kata-qemu.yaml
185+
```
186+
187+
This may take a few moments if the container image needs to be pulled down into the cluster.
188+
Check progress using:
189+
190+
```sh
191+
$ kubectl rollout status deployment php-apache-kata-qemu
192+
```
193+
194+
There are a couple of ways to verify it is running with Kata Containers.
195+
In theory, you should not be able to tell your pod is running as a Kata Containers container.
196+
Careful examination can verify your pod is in fact a Kata Containers pod.
197+
198+
First, look on the node for a `qemu` running. You should see a QEMU command line output here,
199+
indicating that your pod is running inside a Kata Containers VM:
200+
201+
```sh
202+
$ minikube ssh -- pgrep -a qemu
203+
```
204+
205+
Another way to verify Kata Containers is running is to look in the container itself and check
206+
which kernel is running there. For a normal software container you will be running
207+
the same kernel as the node. For a Kata Container you will be running a Kata Containers kernel
208+
inside the Kata Containers VM.
209+
210+
First, examine which kernel is running inside the Minikube node itself:
211+
212+
```sh
213+
$ minikube ssh -- uname -a
214+
```
215+
216+
And then compare that against the kernel that is running inside the container:
217+
218+
```sh
219+
$ podname=$(kubectl get pods -o=name | fgrep php-apache-kata-qemu | sed 's?pod/??')
220+
$ kubectl exec ${podname} -- uname -a
221+
```
222+
223+
You should see the node and pod are running different kernel versions.
224+
225+
## Wrapping up
226+
227+
This guide has shown an easy way to setup Minikube with Kata Containers.
228+
Be aware, this is only a small single node Kubernetes cluster running under a nested virtualization setup.
229+
As such, it has limitations, but as a first introduction to Kata Containers, and how to install it under Kubernetes,
230+
it should suffice for initial learning and experimentation.
231+

0 commit comments

Comments
 (0)