# Setting up kubernetes Cluster

# Installing the Nginx Load Balancer

First we need to install the nginx load balancer with a configuration that points to the two kubernetes Controllers.# Installing docker and kublet on the systems

We need to install docker and kublet on all the systems that are going to be participating in the k8 Cluster# Installing setting up the first k8 Controller

Once the first controller gets installed the others will fall right in line since the configuration comes from the first controller.# Setting up kubernetes on Rocky Linux

## Install kubernetes cluster and nginx

#### What is needed and how they will be used

We will be using 2 control plan nodes, 4 worker/agent nodes and eventually 3 nodes for storage. One will also be used as a reverse proxy using Nginx.

The DNS server that will be used is currently running on a raspberry pi, it is running PiHole and has a local DNS server.

This is an initial install of Kubernetes on Rocky Linux 8.5.

- 1 x Nginx proxy running Rocky Linux 8.5, 2CPU, 4 GB RAM, 32 GB disk
- 2 x VM's for control plane Rocky Linux 8.5, 2CPU, 4 GB RAM, 32 GB disk
- 4 x VM's for Worker nodes Rocky Linux 8.5, 2CPU, 4 GB RAM, 32 GB disk
- 3 x VM's for storage nodes Rocky Linux 8.5, 2CPU, 4 GB RAM, 32 GB disk

<table border="1" id="bkmrk-hostname-ip-address-" style="border-collapse: collapse; width: 87.9012%; height: 327.766px;"><tbody><tr style="height: 29.7969px;"><td style="width: 38.8186%; height: 29.7969px;">**Hostname**</td><td style="width: 17.0151%; height: 29.7969px;">IP Address</td><td style="width: 44.1663%; height: 29.7969px;">**Description**</td></tr><tr style="height: 29.7969px;"><td style="width: 38.8186%; height: 29.7969px;">nlb.binglab.lan</td><td style="width: 17.0151%; height: 29.7969px;">10.14.1.80</td><td style="width: 44.1663%; height: 29.7969px;">Load balancer </td></tr><tr style="height: 29.7969px;"><td style="width: 38.8186%; height: 29.7969px;">mb1prrkubctl001.binglab.lan</td><td style="width: 17.0151%; height: 29.7969px;">10.14.1.81</td><td style="width: 44.1663%; height: 29.7969px;">Kubernetes Control plane</td></tr><tr style="height: 29.7969px;"><td style="width: 38.8186%; height: 29.7969px;">mb1prrkubctl002.binglab.lan</td><td style="width: 17.0151%; height: 29.7969px;">10.14.1.82</td><td style="width: 44.1663%; height: 29.7969px;">Kubernetes Control plane</td></tr><tr style="height: 29.7969px;"><td style="width: 38.8186%; height: 29.7969px;">mb1prrkubwkr001.binglab.lan</td><td style="width: 17.0151%; height: 29.7969px;">10.14.1.83</td><td style="width: 44.1663%; height: 29.7969px;">Kubernetes Worker node</td></tr><tr style="height: 29.7969px;"><td style="width: 38.8186%; height: 29.7969px;">mb1prrkubwkr002.binglab.lan</td><td style="width: 17.0151%; height: 29.7969px;">10.14.1.84</td><td style="width: 44.1663%; height: 29.7969px;">Kubernetes Worker node</td></tr><tr style="height: 29.7969px;"><td style="width: 38.8186%; height: 29.7969px;">mb1prrkubwkr003.binglab.lan</td><td style="width: 17.0151%; height: 29.7969px;">10.14.1.85</td><td style="width: 44.1663%; height: 29.7969px;">Kubernetes Worker node</td></tr><tr style="height: 29.7969px;"><td style="width: 38.8186%; height: 29.7969px;">mb1prrkubwkr004.binglab.lan</td><td style="width: 17.0151%; height: 29.7969px;">10.14.1.86</td><td style="width: 44.1663%; height: 29.7969px;">Kubernetes Worker node</td></tr><tr style="height: 29.7969px;"><td style="width: 38.8186%; height: 29.7969px;">mb1prrkubsto001.binglab.lan</td><td style="width: 17.0151%; height: 29.7969px;">10.14.1.87</td><td style="width: 44.1663%; height: 29.7969px;">Kubernetes Storage node</td></tr><tr style="height: 29.7969px;"><td style="width: 38.8186%; height: 29.7969px;">mb1prrkubsto002.binglab.lan</td><td style="width: 17.0151%; height: 29.7969px;">10.14.1.88</td><td style="width: 44.1663%; height: 29.7969px;">Kubernetes Storage node</td></tr><tr style="height: 29.7969px;"><td style="width: 38.8186%; height: 29.7969px;">mb1prrkubsto003.binglab.lan</td><td style="width: 17.0151%; height: 29.7969px;">10.14.1.89</td><td style="width: 44.1663%; height: 29.7969px;">Kubernetes Storage node</td></tr></tbody></table>

#### Network Diagram

[![k8_layout.jpg](https://bookstack.thebinghamproject.com/uploads/images/gallery/2022-02/scaled-1680-/k8-layout.jpg)](https://bookstack.thebinghamproject.com/uploads/images/gallery/2022-02/k8-layout.jpg)

#### Software/Stack used

- Rocky Linux 8.5
- kubelet
- Kubectl
- Kubernetes-cni
- docker-ce

1 First install the load balancer

1.1 Install package

```
yum install nginx
```

1.2 configure Nginx

```
# For more information on configuration, see:<br></br>#   * Official English Documentation: http://nginx.org/en/docs/<br></br>#   * Official Russian Documentation: http://nginx.org/ru/docs/<br></br><br></br>user nginx;<br></br>worker_processes auto;<br></br>error_log /var/log/nginx/error.log;<br></br>pid /run/nginx.pid;<br></br><br></br># Load dynamic modules. See /usr/share/doc/nginx/README.dynamic.<br></br>include /usr/share/nginx/modules/*.conf;<br></br><br></br>events {<br></br>    worker_connections 1024;<br></br>}<br></br><br></br>stream {<br></br>  upstream k3s_servers {<br></br>    server 10.14.1.81:6443;<br></br>    server 10.14.1.82:6443;<br></br>  }<br></br>  server {<br></br>    listen 6443;<br></br>    proxy_pass k3s_servers;<br></br>  }<br></br>}<br></br>
```

2 Install Docker on the other nodes 6 for now

```
yum update<br></br>yum install -y yum-utils<br></br>sudo yum-config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo<br></br>yum remove docker docker-client docker-client-latest docker-common docker-latest docker-latest-logrotate docker-logrotate docker-engine<br></br>yum install docker-ce docker-ce-cli containerd.io<br></br>systemctl start docker<br></br>systemctl enable docker<br></br>systemctl status docker<br></br>docker ps
```

3 Install Kubernetes but first we need to add the repo on all the nodes

```
cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo<br></br>[kubernetes]<br></br>name=Kubernetes<br></br>baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-\$basearch<br></br>enabled=1<br></br>gpgcheck=1<br></br>repo_gpgcheck=1<br></br>gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg<br></br>exclude=kubelet kubeadm kubectl<br></br>EOF
```

Once the repo is added we need to install the kubernetes packages: on all the nodes

```
dnf install -y kubelet kubeadm kubectl --disableexcludes=kubernetes<br></br>systemctl enable --now kubelet<br></br>echo "KUBELET_EXTRA_ARGS=--cgroup-driver=cgroupfs | sudo tee /etc/sysconfig/kubelet<br></br>systemctl restart kubelet<br></br>systemctl status kubelet<br></br>echo "net.bridge.bridge-nf-call-iptables=1" | sudo tee /etc/sysctl.d/k8s-iptables.conf<br></br>echo "net.bridge.bridge-nf-call-ip6tables=1" | sudo tee /etc/sysctl.d/k8s-ip6tables.conf<br></br>sysctl --system<br></br>
```

4 Install the first control plain node of cluster

```
 kubeadm init --kubernetes-version "1.23.4" --pod-network-cidr "192.168.1.0/16" --service-dns-domain "apps.binglab.lan" --control-plane-endpoint "mb1prrkubnlb001.binglab.lan:6443" --upload-certs
```