[OpenStack] How to Deploy a Kubernetes Cluster Using Kubeadm

This blog presents how to deploy a self-hosted Kubernetes cluster using kubeadm.

Note that usually we use kubeadm to bootstrap a K8s cluster that has several nodes. It is also easy to scale up the created cluster using kubeadm later. However, if you only need to have a learning environment of K8s, it might be easier to use some tools like kind, and minikube: https://kubernetes.io/docs/tasks/tools/

Links to The Relevant Documentations

• Bootstrapping clusters with kubeadm
• Install Docker Engine

I have also made a demonstration video according to the steps in these documents: https://www.youtube.com/watch?v=gvHstbJ7uUo

Before Installing Kubeadm

Let us start this journey by creating OpenStack instances: you could follow the video to create an internal network first, then to create several instances that will host the K8s cluster later. One node is used for hosting the control-plane node. The other nodes are used for the workers.

Configure The Created Instances

Ssh to the created nodes first. Run the following commands on each node.

Letting iptables see bridged traffic

cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
br_netfilter
EOF

cat <<EOF | sudo tee /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
sudo sysctl --system

Installing runtime (e.g., Docker)

We use the installation steps for Ubuntu here. If your instances are using other operating systems, please refer to the Docker installation document here.

• set up the repository

sudo apt-get update
sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg \
    lsb-release

• add Docker’s official GPG key

curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /usr/share/keyrings/docker-archive-keyring.gpg

• set up the stable repository

echo \
  "deb [arch=amd64 signed-by=/usr/share/keyrings/docker-archive-keyring.gpg] https://download.docker.com/linux/ubuntu \
  $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

• install Docker engine

sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io

• post-installation steps for Linux (optional)

Manage Docker as a non-root user:

sudo usermod -aG docker $USER
newgrp docker

Now you should be able to run docker commands without sudo.

Installing kubeadm, kubelet and kubectl

• download the Google Cloud public signing key

sudo curl -fsSLo /usr/share/keyrings/kubernetes-archive-keyring.gpg https://packages.cloud.google.com/apt/doc/apt-key.gpg

• add the Kubernetes apt repository

echo "deb [signed-by=/usr/share/keyrings/kubernetes-archive-keyring.gpg] https://apt.kubernetes.io/ kubernetes-xenial main" | sudo tee /etc/apt/sources.list.d/kubernetes.list

• update apt package index, install kubelet, kubeadm and kubectl, and pin their version

sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl

Creating a cluster with kubeadm

Run the following command on one of the created nodes to initialize it as a control-plane node:

kubeadm init --apiserver-advertise-address=<node's-ip-address> --pod-network-cidr=10.244.0.0/16

The <node's-ip-address> in the command above should be the node's internal IP address, which could be found on the OpenStack dashboard. The value of option --pod-network-cidr has to be 10.244.0.0/16 because we are going to use Flannel as the network add-on: https://github.com/flannel-io/flannel/blob/master/Documentation/kubernetes.md (Depending on which third-party network add-on you choose, option --pod-network-cidr might be optional, or have different values.)

After running this command, you should be able to see some outputs like the following:

Your Kubernetes control-plane has initialized successfully!

...

You can now join any number of machines by running the following on each node
as root:

  kubeadm join ...

Now you can run the kubeadm join ... command on the other nodes to make them join the created K8s cluster.

Making kubectl work for your non-root user

Run the following commands, which are also part of the kubeadm init output:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Installing a Pod network add-on

Here we use Flannel as an example. You only need to install it on the control-plane node:

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

Congrats! Now you have successfully deployed a Kubernetes cluster that you have full control of. You can run something like kubectl get nodes to see if all of your nodes are ready :)