Skip to content

Commit f66c3e9

Browse files
RamLaviRamLavi
committed
blog: Load-balancer for vms on bare-metal K8s clusters
Signed-off-by: Ram Lavi <[email protected]>
1 parent 03ccf75 commit f66c3e9

File tree

1 file changed

+225
-0
lines changed

1 file changed

+225
-0
lines changed

‎_posts/2022-04-03-Virtual-Machines-with-MetalLB.markdown

+225
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,225 @@
1+
---
2+
layout: post
3+
author: Ram Lavi
4+
title: Load-balancer for virtual machines on bare metal Kubernetes clusters
5+
description: This post illustrates setting up a virtual machine with MetalLb loadBalance service.
6+
navbar_active: Blogs
7+
pub-date: April 03
8+
pub-year: 2022
9+
category: news
10+
tags:
11+
[
12+
"Kubevirt",
13+
"kubernetes",
14+
"virtual machine",
15+
"VM",
16+
"load-balancer",
17+
"MetalLB",
18+
]
19+
comments: true
20+
---
21+
22+
## Introduction
23+
24+
Over the last year, Kubevirt has integrated with MetalLB in order to support fault-tolerant access to an application through an external IP address.
25+
As a Cluster administrator using an on-prem cluster without a network load balancer, now it's possible to use MetalLB operator to provide load-balancer capabilities (with Services of type LoadBalancer) to virtual machines.
26+
27+
## Introducing MetalLB
28+
29+
[MetalLB](https://metallb.universe.tf/) allows you to create Kubernetes services of type `LoadBalancer`, and provides network load-balancer implementation in on-prem clusters that don’t run on a cloud provider.
30+
MetalLB is responsible for assigning/unassigning an external IP Address to your service, using IPs from pre-configured pools. In order for the external IPs to be announced externally, MetalLB works in 2 modes: Layer 2 and BGP:
31+
32+
- Layer 2 mode (ARP/NDP)
33+
This mode - which actually does not implement real Load-balancing behavior - provides a failover mechanism where a single node owns the `LoadBalancer` service, until it fails, triggering another node to be chosen as the service owner. This configuration mode makes the IPs reachable from the local network.
34+
In this method, the MetalLB speaker pod announces the IPs in ARP (for Ipv4) and NDP (for Ipv6) protocols over the host network. From a network perspective, the node owning the service appears to have multiple IP addresses assigned to a network interface. After traffic is routed to the node, the service proxy sends the traffic to the application pods.
35+
36+
- BGP mode
37+
This mode provides real load-balancing behavior, by establishing BGP peering sessions with the network routers - which advertise the external IPs of the `LoadBalancer` service, distributing the load over the nodes.
38+
39+
To read more on MetalLB concepts, implementation and limitations, please read [its documentation](https://metallb.universe.tf/concepts/).
40+
41+
## Demo: virtual machine with external IP and MetalLB load balancer
42+
43+
With the following recipe we will end up with a nginx server running on a virtual machine, accessible outside the cluster using MetalLB load balancer.
44+
45+
### Demo environment setup
46+
47+
We are going to use [kind](https://kind.sigs.k8s.io) provider as an ephemeral Kubernetes cluster.
48+
To start it up follow this [guide](https://kind.sigs.k8s.io/docs/user/quick-start/#installation).
49+
50+
#### Prerequirements
51+
52+
- You should have `cluster-admin` privileges on the cluster.
53+
- Getting IP Address pools allocation for MetalLB depends on your environment:
54+
- If you're running a bare-metal cluster in a shared host environment, you need to first reserve this IP Address pool from your hosting provider.
55+
- Alternatively, if you're running on a private cluster, you can use one of the private IP Address spaces (a.k.a RFC1918 addresses). Such addresses are free, and work fine as long as you’re only providing cluster services to your LAN.
56+
57+
### Installing components
58+
59+
#### Installing MetalLB on the cluster
60+
61+
There are many ways to install MetalLB. For the sake of this example, we will install MetalLB operator on the cluster. To do this, please follow this [link](https://operatorhub.io/operator/metallb-operator).
62+
You can confirm the operator is installed by entering the following command (it may take a few seconds for the csv to appear):
63+
```bash
64+
kubectl get csv -n my-metallb-operator \
65+
-o custom-columns=Name:.metadata.name,Phase:.status.phase
66+
```
67+
Example output
68+
```bash
69+
Name Phase
70+
metallb-operator.v0.12.0 Succeeded
71+
```
72+
73+
After the operator in installed, we will create a MetalLB CR:
74+
```yaml
75+
cat <<EOF | kubectl apply -f -
76+
apiVersion: metallb.io/v1beta1
77+
kind: MetalLB
78+
metadata:
79+
name: metallb
80+
namespace: my-metallb-operator
81+
EOF
82+
```
83+
84+
Now let's set an AddressPool. In our specific example we will create the following Layer 2 address pool:
85+
86+
```yaml
87+
cat <<EOF | kubectl apply -f -
88+
apiVersion: metallb.io/v1beta1
89+
kind: AddressPool
90+
metadata:
91+
name: addresspool-sample1
92+
namespace: my-metallb-operator
93+
spec:
94+
protocol: layer2
95+
addresses:
96+
- 172.18.1.1-172.18.1.16
97+
EOF
98+
```
99+
> Note: Since this demo is using a kind cluster, we want this range to be on the docker kind network. For more information, see [link](https://kind.sigs.k8s.io/docs/user/loadbalancer/)
100+
101+
#### Installing Kubevirt on the cluster
102+
103+
Following Kubevirt [user guide](https://kubevirt.io/user-guide/operations/installation/#installing-kubevirt-on-kubernetes) to install released version v0.51.0
104+
```bash
105+
export RELEASE=v0.51.0
106+
kubectl apply -f "https://github.com/kubevirt/kubevirt/releases/download/${RELEASE}/kubevirt-operator.yaml"
107+
kubectl apply -f "https://github.com/kubevirt/kubevirt/releases/download/${RELEASE}/kubevirt-cr.yaml"
108+
kubectl -n kubevirt wait kv kubevirt --timeout=360s --for condition=Available
109+
```
110+
111+
Now we have a Kubernetes cluster with all the pieces to start the Demo.
112+
113+
### Spin up a Virtual Machine running Nginx
114+
115+
Now it's time to start-up a virtual machine running nginx using this yaml:
116+
The virtual machine has a `metallb-service=nginx` we created to use when creating the service.
117+
```yaml
118+
cat <<EOF | kubectl apply -f -
119+
apiVersion: kubevirt.io/v1
120+
kind: VirtualMachine
121+
metadata:
122+
name: fedora-nginx
123+
namespace: default
124+
labels:
125+
metallb-service: nginx
126+
spec:
127+
running: true
128+
template:
129+
metadata:
130+
labels:
131+
metallb-service: nginx
132+
spec:
133+
domain:
134+
devices:
135+
disks:
136+
- disk:
137+
bus: virtio
138+
name: containerdisk
139+
- disk:
140+
bus: virtio
141+
name: cloudinitdisk
142+
interfaces:
143+
- masquerade: {}
144+
name: default
145+
resources:
146+
requests:
147+
memory: 1024M
148+
networks:
149+
- name: default
150+
pod: {}
151+
terminationGracePeriodSeconds: 0
152+
volumes:
153+
- containerDisk:
154+
image: kubevirt/fedora-cloud-container-disk-demo
155+
name: containerdisk
156+
- cloudInitNoCloud:
157+
userData: |-
158+
#cloud-config
159+
password: fedora
160+
chpasswd: { expire: False }
161+
packages:
162+
- nginx
163+
runcmd:
164+
- [ "systemctl", "enable", "--now", "nginx" ]
165+
name: cloudinitdisk
166+
EOF
167+
```
168+
169+
### Expose the virtual machine with a typed LoaBalancer service
170+
171+
When creating the `LoadBalancer` typed service, we need to remember annotating the address-pool we want to use
172+
`addresspool-sample1` and also add the selector `metallb-service: nginx`
173+
174+
```yaml
175+
cat <<EOF | kubectl apply -f -
176+
kind: Service
177+
apiVersion: v1
178+
metadata:
179+
name: metallb-nginx-svc
180+
namespace: default
181+
annotations:
182+
metallb.universe.tf/address-pool: addresspool-sample1
183+
spec:
184+
externalTrafficPolicy: Local
185+
ipFamilies:
186+
- IPv4
187+
ports:
188+
- name: tcp-5678
189+
protocol: TCP
190+
port: 5678
191+
targetPort: 80
192+
type: LoadBalancer
193+
selector:
194+
metallb-service: nginx
195+
EOF
196+
```
197+
198+
Notice that the service got assigned with an external IP from the range assigned by the PoolAddress:
199+
200+
```bash
201+
kubectl get service -n default metallb-nginx-svc
202+
```
203+
204+
Example output
205+
```bash
206+
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
207+
metallb-nginx-svc LoadBalancer 10.96.254.136 172.18.1.1 5678:32438/TCP 53s
208+
```
209+
210+
### Access the virtual machine from outside the cluster
211+
212+
Finally, we can check that the nginx server is accessible from outside the cluster:
213+
```bash
214+
curl -s -o /dev/null 172.18.1.1:5678 && echo "URL exists"
215+
```
216+
217+
Example output
218+
```bash
219+
URL exists
220+
```
221+
222+
## Conclusion
223+
224+
In this blog post we used MetalLB to expose a service using an external IP assigned to a virtual machine.
225+
This illustrates how virtual machine traffic can be load-balanced via a service.

0 commit comments

Comments
 (0)