Intro

Setting up Kubernetes on bare-metal is way more complicated than using Cloud services. Here is how.

Step 1.

Setup at least 2 Ubuntu servers. Feel Free to Try this in a VM First. Designate one server the Master and the rest slaves. Set The Hostnames Appropriately to Simplify the Process.

Step 2. Install Docker

Run this on all servers.
It will install and enable Docker.

sudo apt install docker.io
sudo systemctl enable docker

Step 3. Install Kubernetes

Run this on all servers.
Get all available Kubernetes versions.

curl -s https://packages.cloud.google.com/apt/dists/kubernetes-xenial/main/binary-amd64/Packages | grep Version | awk '{print $2}'

Run the rest of this step on all servers.
This will add the Kubernetes package repository (yes there is only xenial).

curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add
sudo apt-add-repository "deb http://apt.kubernetes.io/ kubernetes-xenial main"

Install a specific version of Kubernetes.
I will use 1.15 because support for 1.16 is a joke.

sudo apt install -qy kubelet=1.15.4-00 kubectl=1.15.4-00  kubeadm=1.15.4-00

Disable swap on all servers or Kubernetes wont function properly

sudo swapoff -a

To sisable swap permanently edit this File and remove the swap line.

nano etc/fstab.conf

Step 4. Initialize a new Cluster

Only do this step on the Master server.
The pod-network-cidr is important for your pod networking.
This cidr is for flannel, it is the simplest one. If you want to use a different network do Your Research here.
Replace [hostipaddress] with the IP the Master server is reachable by from other nodes.

sudo kubeadm init --pod-network-cidr=10.244.0.0/16 --apiserver-advertise-address=[hostipaddress]

This will run for a while. But you will need the output of the command so save it for later.

Step 5. Setup Kubectl

Only do this step on the Master server.
Run these commands as the user who will be using kubectl.

mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config

You can test if this worked by running.

kubectl get nodes

If this complains then you screwed up.

Step 6. Setup Pod Networking

Only do this step on the Master server.
This will setup Flannel for pod networking if you want a different network then go to step 4.

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

Step 7. Join the Slaves to The Cluster

Only do this step on the Slave servers.
Here you need the output from Step 4. It will contain a command that looks like this. (The original command will only work for 24h)

kubeadm join 10.40.0.20:6443 --token kt99k3.ba687tj0drm7zkyc --discovery-token-ca-cert-hash sha256:356890820c089118e48c4f1678bb3382e3ef2b183a7c7d828aac8a0ef70fb9bb

Once you have done this on all slave servers you can go to the Master and run this to check if all slaves have connected.

kubectl get nodes

If this works it will look like this.

NAME          STATUS   ROLES    AGE     VERSION
k8smaster01   Ready    master   5d19h   v1.15.4
k8sworker01   Ready    <none>   5d19h   v1.15.4
k8sworker02   Ready    <none>   5d19h   v1.15.4

If you need to add a new slave after 24h then use this command to get a new join command.

kubeadm token create --print-join-command

Step 8. Done!

You now have a working cluster. If you want a Dashboard, Loadbalancer or Ingress then you need to do some additional work.
If you need to debug something use this.

kubectl run --generator=run-pod/v1 busybox --rm -ti --image=busybox -- /bin/sh

The following steps are only on the master node.

Step 9. Setup XRDP (Optional)

If You don’t want to expose you Dashboard to the scary internet then use this to get a Desktop on your server. This is also great for debugging.

sudo apt-get install xrdp xfce4 xfce4-terminal
echo xfce4-session >~/.xsession
sudo systemctl enable xrdp
sudo service xrdp start

You can now use RDP to connect to Your server. You might need to adjust your Firewall. The normal browser will probably not work so install Firefox through RDP.

sudo apt install firefox

Now You have a fully functional Desktop!

Step 10. Setup Dashboard (Optional)

Apply the Dashboard.

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/recommended/kubernetes-dashboard.yaml

Startup the proxy in a screen to access the dashboard. Only necessary for local access.

screen -dm kubectl proxy

The Dashboard is now accessible in XRDP here.

http://localhost:8001/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:/proxy/

Step 11. Dashboard Token (Optional)

You might have noticed that you need to log into the Dashboard.
This will create a secret.

kubectl create serviceaccount dashboard-admin-sa
kubectl create clusterrolebinding dashboard-admin-sa --clusterrole=cluster-admin --serviceaccount=default:dashboard-admin-sa

Use this to list all secrets.

kubectl get secrets

Run this to Get your login token. But you have to replace the dashboard-admin-sa-token-6rngm with your secret name.

kubectl describe secret dashboard-admin-sa-token-6rngm

Step 12. Setup Metal Lb as a Loadbalancer with layer2 (Optional)

Apply the Metal Lb Config.

kubectl apply -f https://raw.githubusercontent.com/google/metallb/v0.8.3/manifests/metallb.yaml

Create a Config map for Metal Lb

nano metallb-configmap.yaml

And add this to your Config. You will need to to adjust the addresses field it is the range of IP s that Metal Lb can give to services.

apiVersion: v1
kind: ConfigMap
metadata:
  namespace: metallb-system
  name: config
data:
  config: |
    address-pools:
    - name: default
      protocol: layer2
      addresses:
      - 10.40.0.50-10.40.0.150

Now apply your config.

kubectl apply -f metallb-configmap.yaml

Step 13. Set the Dashboard to use the Loadbalcancer (Optional)

Download the dashboard Config.

wget https://raw.githubusercontent.com/kubernetes/dashboard/v1.10.1/src/deploy/recommended/kubernetes-dashboard.yaml

Edit the Config.

nano kubernetes-dashboard.yaml

Under “spec:” add “type: LoadBalancer”. And apply the new Config.

kubectl apply -f kubernetes-dashboard.yaml

To see Where your dashboard is now.

kubectl get svc kubernetes-dashboard -n kube-system

In a browser go to https:// EXTERNAL-IP

Step 13. Setup Gitlab integration (Optional)

In Gitlab go the The Admin Panel. There Should be a Kubernetes menu.
Click Add Kubernetes cluster
Give it a Name.\ Set the API URL to https://[master server ip]:6443/ Use this to list all secrets.

kubectl get secrets

Use the default secret for the cert.

kubectl get secret <secret name> -o jsonpath="{['data']['ca\.crt']}" | base64 --decode

You can now put the Output into the CA field. Now make a secret for Gitlab.

nano gitlab-admin-service-account.yaml

And add this. If you use the gitlab docker image registry than you need to make an extra or remove imagePullSecrets from this Config.

apiVersion: v1
kind: ServiceAccount
metadata:
  name: gitlab-admin
  namespace: kube-system
imagePullSecrets:
- name: registry-gitlab-key
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: gitlab-admin
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: gitlab-admin
  namespace: kube-system

Now you can apply it.

kubectl apply -f gitlab-admin-service-account.yaml

Step 14. Setup Gitlab imagePullSecrets (Optional)

//TODO

Step 15. Setup Traefik Ingress (Optional)