This documents describes how to setup kubernetes from scratch on your own nodes, without using a managed service. This setup uses kubeadm to install and configure kubernetes cluster.
Kubernetes is an open-source system for automating deployment, scaling, and management of containerized applications.
The below steps are applicable for the below mentioned OS
| OS | Version | Codename |
|---|---|---|
| Ubuntu | 16.04 | Xenial |
On all nodes which would be part of this cluster, you need to do the base setup as described in the following steps. To simplify this, you could also download and run installkube.sh script
We need to create a repository to download Kubernetes.
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
cat <<EOF > /etc/apt/sources.list.d/kubernetes.list
deb http://apt.kubernetes.io/ kubernetes-xenial main
EOF
We should update the machines before installing so that we can update the repository.
apt-get update -y
Installing all the packages with dependencies:
apt-get install -y docker.io kubelet kubeadm kubectl kubernetes-cni
rm -rf /var/lib/kubelet/*
In order for many container networks to work, the following needs to be enabled on each node.
sysctl net.bridge.bridge-nf-call-iptables=1
The above steps has to be followed in all the nodes.
This tutorial assumes kube-01 as the master and used kubeadm as a tool to install and setup the cluster. This section also assumes that you are using vagrant based setup provided along with this tutorial. If not, please update the IP address of the master accordingly.
To initialize master, run this on kube-01
kubeadm init --apiserver-advertise-address 192.168.56.101(replace with your Ip Address) --pod-network-cidr=192.168.0.0/16 (replace with your Subnet)
After master being initialized, it should display the command which could be used on all worker/nodes to join the k8s cluster.
e.g.
kubeadm join --token c04797.8db60f6b2c0dd078 192.168.12.10:6443 --discovery-token-ca-cert-hash sha256:88ebb5d5f7fdfcbbc3cde98690b1dea9d0f96de4a7e6bf69198172debca74cd0
Copy and paste it on all node.
If you lose the join token, you could retrieve it using
kubeadm token list
On successfully joining the master, you should see output similar to following,
root@kube-03:~# kubeadm join --token c04797.8db60f6b2c0dd078 159.203.170.84:6443 --discovery-token-ca-cert-hash sha256:88ebb5d5f7fdfcbbc3cde98690b1dea9d0f96de4a7e6bf69198172debca74cd0 [kubeadm] WARNING: kubeadm is in beta, please do not use it for production clusters. [preflight] Running pre-flight checks [discovery] Trying to connect to API Server "159.203.170.84:6443" [discovery] Created cluster-info discovery client, requesting info from "https://159.203.170.84:6443" [discovery] Requesting info from "https://159.203.170.84:6443" again to validate TLS against the pinned public key [discovery] Cluster info signature and contents are valid and TLS certificate validates against pinned roots, will use API Server "159.203.170.84:6443" [discovery] Successfully established connection with API Server "159.203.170.84:6443" [bootstrap] Detected server version: v1.8.2 [bootstrap] The server supports the Certificates API (certificates.k8s.io/v1beta1)Node join complete:
- Certificate signing request sent to master and response received.
- Kubelet informed of new secure connection details.
Run 'kubectl get nodes' on the master to see this machine join.
On Master Node
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
Installing overlay network is necessary for the pods to communicate with each other across the hosts. It is necessary to do this before you try to deploy any applications to your cluster.
There are various overlay networking drivers available for kubernetes. We are going to use Weave Net.
export kubever=$(kubectl version | base64 | tr -d '\n')
kubectl apply -f "https://cloud.weave.works/k8s/net?k8s-version=$kubever"
You could validate the status of this cluster, health of pods and whether all the components are up or not by using a few or all of the following commands.
To check if nodes are ready
kubectl get nodes kubectl get cs
[ Expected output ]
root@kube-01:~# kubectl get nodes
NAME STATUS ROLES AGE VERSION
kube-01 Ready master 9m v1.8.2
kube-02 Ready <none> 4m v1.8.2
kube-03 Ready <none> 4m v1.8.2
Additional Status Commands
kubectl versionkubectl cluster-info
kubectl get pods -n kube-system
kubectl get events
It will take a few minutes to have the cluster up and running with all the services.
- Nodes are node in Ready status
- kube-dns is crashing constantly
root@kube-01:~# kubectl logs kube-dns-545bc4bfd4-dh994 kubedns -n kube-system I1106 14:41:15.542409 1 dns.go:48] version: 1.14.4-2-g5584e04 I1106 14:41:15.543487 1 server.go:70] Using
....
After the Pod networks is installled, We can install another add-on service which is Kubernetes Dashboard.
Installing Dashboard:
kubectl apply -f https://gist.githubusercontent.com/initcron/32ff89394c881414ea7ef7f4d3a1d499/raw/4863613585d05f9360321c7141cc32b8aa305605/kube-dashboard.yaml
This will create a pod for the Kubernetes Dashboard.
To access the Dashboard in th browser, run the below command
kubectl describe svc kubernetes-dashboard -n kube-system
Sample output:
kubectl describe svc kubernetes-dashboard -n kube-system
Name: kubernetes-dashboard
Namespace: kube-system
Labels: app=kubernetes-dashboard
Selector: app=kubernetes-dashboard
Type: NodePort
IP: 10.98.148.82
Port: <unset> 80/TCP
NodePort: <unset> 31000/TCP
Endpoints: 10.40.0.1:9090
Session Affinity: None
Now check for the node port, here it is 31000, and go to the browser, and access the dashboard with the following URL
do not use the IP above, use master node IP instead
http://NODEIP:31000