Skip to content

Commit 08510b7

Browse files
RamLaviRamLavi
committed
blog: Load-balancer for vms on bare-metal K8s clusters
Signed-off-by: Ram Lavi <[email protected]>
1 parent 03ccf75 commit 08510b7

File tree

1 file changed

+228
-0
lines changed

1 file changed

+228
-0
lines changed

‎_posts/2022-04-03-Virtual-Machines-with-MetalLB.markdown

+228
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,228 @@
1+
---
2+
layout: post
3+
author: Ram Lavi
4+
title: Load-balancer for virtual machines on bare metal Kubernetes clusters
5+
description: This post illustrates setting up a virtual machine with MetalLb loadBalance service.
6+
navbar_active: Blogs
7+
pub-date: April 03
8+
pub-year: 2022
9+
category: news
10+
tags:
11+
[
12+
"Kubevirt",
13+
"kubernetes",
14+
"virtual machine",
15+
"VM",
16+
"load-balancer",
17+
"MetalLB",
18+
]
19+
comments: true
20+
---
21+
22+
## Introduction
23+
24+
Over the last year, Kubevirt has integrated with MetalLB in order to support fault-tolerant access to an application through an external IP address.
25+
As a Cluster administrator using an on-prem cluster without a network load balancer, now it's possible to add MetalLB operator and gain a load-balancer capabilities (with Services of type LoadBalancer) to virtual machines.
26+
27+
## Introducing MetalLB
28+
29+
[MetalLB](https://metallb.universe.tf/) allows you to create Kubernetes services of type `LoadBalancer`, and provides network load-balancer implementation in on-prem clusters that don’t run on a cloud provider.
30+
MetalLB is responsible for assigning/unassigning an external IP Address to your service, using IPs from pre-configured pools. In order for the external IPs to be announced externally, MetalLB works in 2 modes: Layer 2 and BGP:
31+
32+
- Layer 2 mode (ARP/NDP)
33+
This mode - which actually does not implement real Load-balancing behavior - provides a failover mechanism where a single node owns the `LoadBalancer` service, until it fails, triggering another node to be chosen as the service owner. This configuration mode makes the IPs reachable from the local network.
34+
In this method, the MetalLB speaker pod announces the IPs in ARP (for Ipv4) and NDP (for Ipv6) protocols over the host network. From a network perspective, the node owning the service appears to have multiple IP addresses assigned to a network interface. After traffic is routed to the node, the service proxy sends the traffic to the application pods.
35+
36+
- BGP mode
37+
This mode provides real load-balancing behavior, by establishing BGP peering sessions with the network routers - which advertise the external IPs of the `LoadBalancer` service, distributing the load over the nodes.
38+
39+
To read more on MetalLB concepts, implementation and limitations, please read [its documentation](https://metallb.universe.tf/concepts/).
40+
41+
## Demo: virtual machine with external IP and MetalLB load balancer
42+
43+
With the following recipe we will end up with a nginx server running on a virtual machine, accessible outside the cluster using MetalLB load balancer.
44+
45+
### Demo environment setup
46+
47+
We are going to use [kind](https://kind.sigs.k8s.io) provider as an ephemeral Kubernetes cluster.
48+
To start it up follow this [guide](https://kind.sigs.k8s.io/docs/user/quick-start/#installation).
49+
50+
#### Prerequirements
51+
52+
- You should have `cluster-admin` privileges on the cluster.
53+
- IP Address allocation:
54+
How you get IP address pools for MetalLB depends on your environment:
55+
- If you're running a bare-metal cluster in a shared host environment, you need to first reserve this IP Address pool from your hosting provider.
56+
- Alternatively, if you're running on a private cluster, you can use one of the private IP Address spaces (a.k.a RFC1918 addresses). Such addresses are free, and work fine as long as you’re only providing cluster services to your LAN.
57+
58+
### Installing components
59+
60+
#### Installing MetalLB on the cluster
61+
62+
There are many ways to install MetalLB. For the sake of this example, we will install MetalLB operator on the cluster. To do this, please follow this [link](https://operatorhub.io/operator/metallb-operator).
63+
You can confirm the operator is installed by entering the following command:
64+
```bash
65+
kubectl get csv -n my-metallb-operator \
66+
-o custom-columns=Name:.metadata.name,Phase:.status.phase
67+
```
68+
Example output
69+
```bash
70+
Name Phase
71+
metallb-operator.v0.12.0 Succeeded
72+
```
73+
74+
After the operator in installed, we will create a MetalLB CR:
75+
```yaml
76+
cat <<EOF | kubectl apply -f -
77+
apiVersion: metallb.io/v1beta1
78+
kind: MetalLB
79+
metadata:
80+
name: metallb
81+
namespace: my-metallb-operator
82+
EOF
83+
```
84+
85+
Now let's set an AddressPool. In our specific example we will create the following Layer 2 address pool:
86+
87+
```yaml
88+
cat <<EOF | kubectl apply -f -
89+
apiVersion: metallb.io/v1beta1
90+
kind: AddressPool
91+
metadata:
92+
name: addresspool-sample1
93+
namespace: my-metallb-operator
94+
spec:
95+
protocol: layer2
96+
addresses:
97+
- 172.18.1.1-172.18.1.16
98+
EOF
99+
```
100+
> Note: Since this demo is using a kind cluster, we want this range to be on the docker kind network. For more information, see [link](https://kind.sigs.k8s.io/docs/user/loadbalancer/)
101+
102+
#### Installing Kubevirt on the cluster
103+
104+
Following Kubevirt [user guide](https://kubevirt.io/user-guide/operations/installation/#installing-kubevirt-on-kubernetes) to install released version v0.51.0
105+
```bash
106+
export RELEASE=v0.51.0
107+
kubectl apply -f "https://github.com/kubevirt/kubevirt/releases/download/${RELEASE}/kubevirt-operator.yaml"
108+
kubectl apply -f "https://github.com/kubevirt/kubevirt/releases/download/${RELEASE}/kubevirt-cr.yaml"
109+
kubectl -n kubevirt wait kv kubevirt --timeout=360s --for condition=Available
110+
```
111+
112+
Now we have a Kubernetes cluster with all the pieces to start the Demo.
113+
114+
### Spin up a Virtual Machine running Nginx
115+
116+
Now it's time to start-up a virtual machine running nginx using this yaml:
117+
The virtual machine has a `metallb-service=nginx` we created to use when creating the service.
118+
```yaml
119+
cat <<EOF | kubectl apply -f -
120+
apiVersion: kubevirt.io/v1
121+
kind: VirtualMachine
122+
metadata:
123+
name: fedora-nginx
124+
namespace: default
125+
labels:
126+
metallb-service: nginx
127+
spec:
128+
running: true
129+
template:
130+
metadata:
131+
labels:
132+
metallb-service: nginx
133+
spec:
134+
domain:
135+
devices:
136+
disks:
137+
- disk:
138+
bus: virtio
139+
name: containerdisk
140+
- disk:
141+
bus: virtio
142+
name: cloudinitdisk
143+
interfaces:
144+
- masquerade: {}
145+
name: default
146+
resources:
147+
requests:
148+
memory: 1024M
149+
networks:
150+
- name: default
151+
pod: {}
152+
terminationGracePeriodSeconds: 0
153+
volumes:
154+
- containerDisk:
155+
image: kubevirt/fedora-cloud-container-disk-demo
156+
name: containerdisk
157+
- cloudInitNoCloud:
158+
userData: |-
159+
#cloud-config
160+
password: fedora
161+
chpasswd: { expire: False }
162+
packages:
163+
- nginx
164+
runcmd:
165+
- [ "systemctl", "enable", "--now", "nginx" ]
166+
name: cloudinitdisk
167+
EOF
168+
```
169+
170+
### Expose the virtual machine with a typed LoaBalancer service
171+
172+
When creating the `LoadBalancer` typed service, we need to remember annotating the address-pool we want to use
173+
`addresspool-sample1` and also add the selector `metallb-service: nginx`
174+
175+
```yaml
176+
cat <<EOF | kubectl apply -f -
177+
kind: Service
178+
apiVersion: v1
179+
metadata:
180+
name: metallb-nginx-svc
181+
namespace: default
182+
annotations:
183+
metallb.universe.tf/address-pool: addresspool-sample1
184+
spec:
185+
externalTrafficPolicy: Local
186+
ipFamilies:
187+
- IPv4
188+
ports:
189+
- name: tcp-5678
190+
protocol: TCP
191+
port: 5678
192+
targetPort: 80
193+
allocateLoadBalancerNodePorts: true
194+
type: LoadBalancer
195+
ipFamilyPolicy: SingleStack
196+
selector:
197+
metallb-service: nginx
198+
EOF
199+
```
200+
201+
Notice that the service got assigned with an external IP from the range assigned by the PoolAddress:
202+
203+
```bash
204+
kubectl get service -n default metallb-nginx-svc
205+
```
206+
207+
Example output
208+
```bash
209+
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
210+
metallb-nginx-svc LoadBalancer 10.96.254.136 172.18.1.1 5678:32438/TCP 53s
211+
```
212+
213+
### Access the virtual machine from outside the cluster
214+
215+
Finally, we can check that the nginx server is accessible from outside the cluster:
216+
```bash
217+
curl -s -o /dev/null 172.18.1.1:5678 && echo "URL exists"
218+
```
219+
220+
Example output
221+
```bash
222+
URL exists
223+
```
224+
225+
## Conclusion
226+
227+
In this blog post we used MetalLB to expose a service using an external IP assigned to a virtual machine.
228+
This illustrates how virtual machine traffic can be load-balanced via a service.

0 commit comments

Comments
 (0)