Install LoadBalancer for Kubernetes for On-Prem or Home-Lab

If you are having a Kubernetes cluster running in an on-prem environment, unlike managed Kubernetes solutions you must be missing the fun of having an automatic assignment of load balancer IP to your exposed services. No more pending external IP 🙂

The solution is using an on-prem load balancer. This really beefs up the capabilities of the ingress controller for external access. This short post will show how to set up and use metallb as your load balancer.

Whenever you expose any service by a “LoadBalancer” type, metallb will automatically assign an IP to the load balancer. However, we need to tell metallb the source/pool of the IP address to use. Once we provide the available pool, metallb will assign any new load balancer request from that pool of IP addresses.

Preparation: Before installation ensure the Kube-proxy mode


If you’re using Kube-proxy in IPVS mode since Kubernetes v1.14.2 you have to enable strict ARP mode. This limitation is listed in the official preparation section of the installation guide. For example, using the below command I figured out that I am using Kube-proxy in “ipvs” mode, but the strictARP is set to false. This means I need to set the strictARP mode to true.

You may run the following command to figure out your Kube-proxy configuration.

 kubectl get cm -n kube-system  kube-proxy  -o yaml |grep -E 'mode:|strictARP:'
      strictARP: false
    mode: ipvs


Make the changes if you see a result like the one in the above snippet.

#take the backup of the configuration, Just in case you want to revert back
kubectl get cm -n kube-system kube-proxy -o yaml > kube-proxy.bkp

#edit the config map
kubectl get cm -n kube-system  kube-proxy  -o yaml |sed -r 's/strictARP: false/strictARP: true/g'  |kubectl replace  -f  - --force

#verify the changes
kubectl get cm -n kube-system  kube-proxy  -o yaml |grep -E 'mode:|strictARP:'
      strictARP: true
    mode: ipvs


Step-1 Create a Pool of IPs that are available for metallb


Currently, I am installing metallb on my home lab, I will be using my router interface eno2. For you, this interface might differ. For example, if you are using wifi, you might want to use an interface similar to wlo1.

ip addr sh  eno2
2: eno2: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether ff brd ff:ff:ff:ff:ff:ff
    altname enp0s31f6
    inet 192.168.1.135/24 brd 192.168.1.255 scope global dynamic noprefixroute eno2
       valid_lft 84235sec preferred_lft 84235sec


In the below output, I get the 192.168.1.135/24 which can be used for assigning the load balancer IP. You can use the entire network. OR better is to pick the IP from the last of this subnet.
For example, you can choose 192.168.1.240 to 192.168.1.250, or any IP range towards the last to ensure that these IP addresses are not already assigned to some other node in the network.

ipcalc 192.168.1.135/24
Address:   192.168.1.135        11000000.10101000.00000001. 10000111
Netmask:   255.255.255.0 = 24   11111111.11111111.11111111. 00000000
Wildcard:  0.0.0.255            00000000.00000000.00000000. 11111111
=>
Network:   192.168.1.0/24       11000000.10101000.00000001. 00000000
HostMin:   192.168.1.1          11000000.10101000.00000001. 00000001
HostMax:   192.168.1.254        11000000.10101000.00000001. 11111110
Broadcast: 192.168.1.255        11000000.10101000.00000001. 11111111
Hosts/Net: 254                   Class C, Private Internet


Step-2: Download the helm chart

helm repo add metallb https://metallb.github.io/metallb
helm repo update


Step-3: Dump the values.yml of the helm chart

helm show values metallb/metallb > values_metallb.yml


Step-4: Add the Range of IP addresses available to Metallb


Add the following contents to your values.yml file under configInline section. Note the IP range should be the one based on the values decided in Step 1.

configInline:
  address-pools:
   - name: default
     protocol: layer2
     addresses:
     - 192.168.1.240-192.168.1.250


Step-5: Perform Installation

helm install metallb metallb/metallb -f values_metallb.yml   --namespace metallb-system --create-namespace


Step-6: Validate that all the resources in the metallb-system namespace are up and running.

kubectl get all -n metallb-system 
NAME                                      READY   STATUS    RESTARTS   AGE
pod/metallb-controller-6c9749d779-p47bd   1/1     Running   0          3m44s
pod/metallb-speaker-7slcr                 1/1     Running   0          3m44s
pod/metallb-speaker-cbdqk                 1/1     Running   0          3m44s
pod/metallb-speaker-jrxvv                 1/1     Running   0          3m44s
pod/metallb-speaker-m7xhg                 1/1     Running   0          3m44s

NAME                             DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
daemonset.apps/metallb-speaker   4         4         4       4            4           kubernetes.io/os=linux   3m44s

NAME                                 READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/metallb-controller   1/1     1            1           3m44s

NAME                                            DESIRED   CURRENT   READY   AGE
replicaset.apps/metallb-controller-6c9749d779   1         1         1       3m44s


Step-7: Installation is completed and time to test the installation. [OPTIONAL]


Here I have created a deployment and exposed it by a LoadBalancer type of service. Note that an External IP address is assigned. Especially this IP is the first one in the IP range we provided during the time of installation.

kubectl create  deployment  web-server --image nginx
kubectl expose  deployment  web-server  --type LoadBalancer --port 80



kubectl get svc
NAME         TYPE           CLUSTER-IP      EXTERNAL-IP     PORT(S)        AGE
kubernetes   ClusterIP      10.233.0.1      <none>          443/TCP        52s
web-server   LoadBalancer   10.233.39.201   192.168.1.240   80:31318/TCP   4s


Also, the External IP is accessible from outside of the cluster. (from my workstation or from the web browser of any device in the same network)

curl 192.168.1.240
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>


Step-8: Now let’s try to exhaust the IP range and see the behavior of metallb [OPTIONAL]


This is an optional step to see, what happens when the users are requesting more load balancer IP than available in the pool. In the below example, it’s clear that if the pool is exhausted then any new request will remain in a pending state.

kubectl expose  deployment  web-server  --type LoadBalancer --port 80 --name web-server-2
service/web-server-2 exposed

kubectl expose  deployment  web-server  --type LoadBalancer --port 80 --name web-server-3
service/web-server-3 exposed


kubectl expose  deployment  web-server  --type LoadBalancer --port 80 --name web-server-4
service/web-server-4 exposed

kubectl expose  deployment  web-server  --type LoadBalancer --port 80 --name web-server-5
service/web-server-5 exposed


kubectl expose  deployment  web-server  --type LoadBalancer --port 80 --name web-server-6
service/web-server-6 exposed


kubectl expose  deployment  web-server  --type LoadBalancer --port 80 --name web-server-7
service/web-server-7 exposed


kubectl expose  deployment  web-server  --type LoadBalancer --port 80 --name web-server-8
service/web-server-8 exposed


kubectl expose  deployment  web-server  --type LoadBalancer --port 80 --name web-server-9
service/web-server-9 exposed


kubectl expose  deployment  web-server  --type LoadBalancer --port 80 --name web-server-10
service/web-server-10 exposed


kubectl expose  deployment  web-server  --type LoadBalancer --port 80 --name web-server-11
service/web-server-11 exposed


kubectl expose  deployment  web-server  --type LoadBalancer --port 80 --name web-server-12
service/web-server-12 exposed

kubectl get svc
NAME            TYPE           CLUSTER-IP      EXTERNAL-IP     PORT(S)        AGE
kubernetes      ClusterIP      10.233.0.1      <none>          443/TCP        6m46s
web-server      LoadBalancer   10.233.39.201   192.168.1.240   80:31318/TCP   5m58s
web-server-10   LoadBalancer   10.233.37.67    192.168.1.249   80:31156/TCP   20s
web-server-11   LoadBalancer   10.233.41.115   192.168.1.250   80:30710/TCP   18s
web-server-12   LoadBalancer   10.233.51.253   <pending>       80:31778/TCP   1s #<---Pending!
web-server-2    LoadBalancer   10.233.40.15    192.168.1.241   80:31799/TCP   35s
web-server-3    LoadBalancer   10.233.60.101   192.168.1.242   80:31392/TCP   34s
web-server-4    LoadBalancer   10.233.29.121   192.168.1.243   80:32409/TCP   33s
web-server-5    LoadBalancer   10.233.40.150   192.168.1.244   80:30387/TCP   32s
web-server-6    LoadBalancer   10.233.53.114   192.168.1.245   80:32704/TCP   31s
web-server-7    LoadBalancer   10.233.45.74    192.168.1.246   80:31783/TCP   31s
web-server-8    LoadBalancer   10.233.62.72    192.168.1.247   80:31479/TCP   30s
web-server-9    LoadBalancer   10.233.12.193   192.168.1.248   80:31284/TCP   28s


Note: About deleting the load balancer services

Do not use –force while deleting the services, this would prevent releasing the IP assigned to previous services and new services would remain in the pending state as the lease for the previous allocation will not clear completely.

kubectl delete  svc --all --force 
warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
service "kubernetes" force deleted
service "web-server" force deleted
service "web-server-10" force deleted
service "web-server-11" force deleted
service "web-server-12" force deleted
service "web-server-2" force deleted
service "web-server-3" force deleted
service "web-server-4" force deleted
service "web-server-5" force deleted
service "web-server-6" force deleted
service "web-server-7" force deleted
service "web-server-8" force deleted
service "web-server-9" force deleted


kubectl expose  deployment  web-server  --type LoadBalancer --port 80 --name web-server-13
service/web-server-13 exposed


kubectl get svc
NAME            TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
kubernetes      ClusterIP      10.233.0.1      <none>        443/TCP        3s
web-server-13   LoadBalancer   10.233.21.243   <pending>     80:31437/TCP   1s



Summary:

Reference: https://metallb.universe.tf/installation/

Leave a Comment

Your email address will not be published.

Scroll to Top