Skip to content

Commit f2495f5

Browse files
RamLaviRamLavi
committed
blog: Load-balancer for vms on bare-metal K8s clusters
Signed-off-by: Ram Lavi <[email protected]>
1 parent 03ccf75 commit f2495f5

File tree

1 file changed

+220
-0
lines changed

1 file changed

+220
-0
lines changed

‎_posts/2022-04-03-Virtual-Machines-with-MetalLB.markdown

+220
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,220 @@
1+
---
2+
layout: post
3+
author: ralavi
4+
title: Load-balancer for virtual machines on bare metal Kubernetes clusters
5+
description: This post illustrates setting up a virtual machine with MetalLb loadBalance service.
6+
navbar_active: Blogs
7+
pub-date: April 03
8+
pub-year: 2022
9+
category: news
10+
tags:
11+
[
12+
Kubevirt,
13+
networking,
14+
service,
15+
load-balancer,
16+
virtual machine,
17+
MetalLB
18+
]
19+
comments: true
20+
---
21+
22+
## Introduction
23+
24+
Over the last year, Kubevirt has integrated with MetalLB in order to support fault-tolerant access to an application through an external IP address. As a Cluster administrator using a bare-metal cluster, now it's possible to add MetalLB operator and gain a load-balancer capabilities (with Services of type LoadBalancer) to virtual machines.
25+
26+
## Introducing MetalLB
27+
28+
[MetalLB](https://metallb.universe.tf/) allows you to create Kubernetes services of type `LoadBalancer`, and provides network load-balancer implementation in clusters that don’t run on a cloud provider, such as bare-metals.
29+
MetalLB is responsible for assigning/unassigning an external IP Address to your service, using IPs from pre-configured pools. In order for the external IPs to be announced externally, MetalLB works in 2 working modes: Layer 2 and BGP modes:
30+
31+
- Layer 2 mode (ARP/NDP)
32+
This mode does not implement Load-balancing behavior provides a failover mechanism where one node owns the `LoadBalancer` service, until the node fails and another node is chosen. In a strict sense, this is not a real load-balancing behavior, but instead it makes the IPs reachable to the local network. This method announces the IPs in ARP (for Ipv4) and NDP (for Ipv6) protocols over the network. From a network perspective, the node appears to have multiple IP addresses assigned to a network interface on the chosen node.
33+
34+
- BGP mode
35+
This mode provides a real load-balancing behavior, By establishing BGP peering sessions with the network routers. They in turn advertise the external IPs of the `LoadBalancer` service, distributing the load over the nodes.
36+
37+
To read more on MetalLB concepts, implementation and limitations, please read [its documentation](https://metallb.universe.tf/concepts/).
38+
39+
## Demo: virtual machine with external IP and MetalLB load balancer
40+
41+
With the following recipe we will end up with a nginx server running on a virtual machine, accessible outside the cluster using MetalLB load balancer.
42+
43+
### Demo environment setup
44+
45+
We are going to use [kind](https://kind.sigs.k8s.io) provider as an ephemeral Kubernetes cluster.
46+
To start it up follow this [guide](https://kind.sigs.k8s.io/docs/user/quick-start/#installation).
47+
48+
### Installing components
49+
50+
#### Installing MetalLB on the cluster
51+
52+
There are many ways to install MetalLB. For the sake of this example, we will install MetalLB operator on the cluster. To do this, please follow this [link](https://operatorhub.io/operator/metallb-operator). Note
53+
that you will need `cluster-admin` privileges.
54+
You can confirm the operator is installed by entering the following command:
55+
```bash
56+
kubectl get csv -n my-metallb-operator \
57+
-o custom-columns=Name:.metadata.name,Phase:.status.phase
58+
```
59+
Example output
60+
```bash
61+
Name Phase
62+
metallb-operator.v0.12.0 Succeeded
63+
```
64+
65+
After the operator in installed, we will create a MetalLB CR:
66+
```yaml
67+
cat <<EOF | kubectl apply -f -
68+
apiVersion: metallb.io/v1beta1
69+
kind: MetalLB
70+
metadata:
71+
name: metallb
72+
namespace: my-metallb-operator
73+
EOF
74+
```
75+
76+
Now let's set an AddressPool. In our specific example we will create the following Layer 2 address pool:
77+
78+
```yaml
79+
cat <<EOF | kubectl apply -f -
80+
apiVersion: metallb.io/v1beta1
81+
kind: AddressPool
82+
metadata:
83+
name: addresspool-sample1
84+
namespace: my-metallb-operator
85+
spec:
86+
protocol: layer2
87+
addresses:
88+
- 172.18.1.1-172.18.1.16
89+
EOF
90+
```
91+
> Notes:
92+
> - If you're running a bare-metal cluster in a colocation factory, you need to first reserve this IP Address pool from your hosting provider. Alternatively, if you're running on a purely private cluster, you can use one of the private IP Address spaces (a.k.a RFC1918 addresses). Such addresses are free, and work fine as long as you’re only providing cluster services to your LAN.
93+
> - Since we are using a kind cluster, we want this range to be on the docker kind network. For more information, see [link](https://kind.sigs.k8s.io/docs/user/loadbalancer/)
94+
95+
#### Installing Kubevirt on the cluster
96+
97+
Following Kubevirt [user guide](https://kubevirt.io/user-guide/operations/installation/#installing-kubevirt-on-kubernetes) to install released version v0.51.0
98+
```bash
99+
export RELEASE=v0.51.0
100+
kubectl apply -f "https://github.com/kubevirt/kubevirt/releases/download/${RELEASE}/kubevirt-operator.yaml"
101+
kubectl apply -f "https://github.com/kubevirt/kubevirt/releases/download/${RELEASE}/kubevirt-cr.yaml"
102+
kubectl -n kubevirt wait kv kubevirt --timeout=360s --for condition=Available
103+
```
104+
105+
Now we have a Kubernetes cluster with all the pieces to start the Demo.
106+
107+
### Spin up a Virtual Machine running Nginx
108+
109+
Now it's time to start-up a virtual machine running nginx using this yaml:
110+
The virtual machine has a `metallb-service=nginx` we created to use when creating the service.
111+
```yaml
112+
cat <<EOF | kubectl apply -f -
113+
apiVersion: kubevirt.io/v1
114+
kind: VirtualMachine
115+
metadata:
116+
name: fedora-nginx
117+
namespace: default
118+
labels:
119+
metallb-service: nginx
120+
spec:
121+
running: true
122+
template:
123+
metadata:
124+
labels:
125+
metallb-service: nginx
126+
spec:
127+
domain:
128+
devices:
129+
disks:
130+
- disk:
131+
bus: virtio
132+
name: containerdisk
133+
- disk:
134+
bus: virtio
135+
name: cloudinitdisk
136+
interfaces:
137+
- masquerade: {}
138+
name: default
139+
resources:
140+
requests:
141+
memory: 1024M
142+
networks:
143+
- name: default
144+
pod: {}
145+
terminationGracePeriodSeconds: 0
146+
volumes:
147+
- containerDisk:
148+
image: kubevirt/fedora-cloud-container-disk-demo
149+
name: containerdisk
150+
- cloudInitNoCloud:
151+
userData: |-
152+
#!/bin/bash
153+
echo "fedora" |passwd fedora --stdin
154+
sudo yum install -y nginx
155+
sudo systemctl enable nginx
156+
sudo systemctl start nginx
157+
name: cloudinitdisk
158+
EOF
159+
```
160+
161+
### Expose the virtual machine with a typed LoaBalancer service
162+
163+
When creating the `LoadBalancer` typed service, we need to remember annotating the address-pool we want to use
164+
`addresspool-sample1` and also add the selector `metallb-service: nginx`
165+
166+
```yaml
167+
cat <<EOF | kubectl apply -f -
168+
kind: Service
169+
apiVersion: v1
170+
metadata:
171+
name: metallb-nginx-svc
172+
namespace: default
173+
annotations:
174+
metallb.universe.tf/address-pool: addresspool-sample1
175+
spec:
176+
externalTrafficPolicy: Local
177+
ipFamilies:
178+
- IPv4
179+
ports:
180+
- name: tcp-5678
181+
protocol: TCP
182+
port: 5678
183+
targetPort: 80
184+
allocateLoadBalancerNodePorts: true
185+
type: LoadBalancer
186+
ipFamilyPolicy: SingleStack
187+
sessionAffinity: None
188+
selector:
189+
metallb-service: nginx
190+
EOF
191+
```
192+
193+
Notice that the service got assigned with an external IP from the range assigned by the PoolAddress:
194+
195+
```bash
196+
kubectl get service -n default metallb-nginx-svc
197+
```
198+
199+
Example output
200+
```bash
201+
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
202+
metallb-nginx-svc LoadBalancer 10.96.254.136 172.18.1.1 5678:32438/TCP 53s
203+
```
204+
205+
### Access the virtual machine from outside the cluster
206+
207+
Finally, we can check that the nginx server is accessible from outside the cluster:
208+
```bash
209+
curl -s -o /dev/null 172.18.1.1:5678 && echo "URL exists"
210+
```
211+
212+
Example output
213+
```bash
214+
URL exists
215+
```
216+
217+
## Conclusion
218+
219+
In this blog post we used MetalLB to expose a service using an external IP assigned to a virtual machine.
220+
This illustrates how virtual machine traffic can be load-balanced via a service.

0 commit comments

Comments
 (0)