Sa Pham

Interested in Cloud Computing, Programming and Security.

Running kubernetes with Openstack Cloud Provider

22 Feb 2018 »

Kubernetes use Openstack as Cloud Provider

  • In this guide, I will setup kubernetes (1.9.3) cluster by using kubeadm and use Openstack as a Cloud Provider

Pros

  • Use Cinder volume for persistent volume
  • Use Octavia or LBaas Agent for Loadbalancer service type

Cons

  • Currently, we have to setup cloud credential manual and change args for kubelet in all host

Install Docker, Kubeadm , kubelet

The simplest way to install docker is use docker.io package from Ubuntu repository,

sudo apt install docker.io

I will use docker 1.13

[email protected]:~/charts/stable/grafana# docker version
Client:
 Version:      1.13.1
 API version:  1.26
 Go version:   go1.6.2
 Git commit:   092cba3
 Built:        Thu Nov  2 20:40:23 2017
 OS/Arch:      linux/amd64

Server:
 Version:      1.13.1
 API version:  1.26 (minimum version 1.12)
 Go version:   go1.6.2
 Git commit:   092cba3
 Built:        Thu Nov  2 20:40:23 2017
 OS/Arch:      linux/amd64
 Experimental: false
  • Install kubeadm, kubelet latest, type following command
apt-get update && apt-get install -y apt-transport-https
curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | apt-key add -
cat << EOF >/etc/apt/sources.list.d/kubernetes.list
deb http://apt.kubernetes.io/ kubernetes-xenial main
EOF
apt-get update
apt-get install -y kubelet kubeadm kubectl
  • After install these tool, we can check other requisites by run this command
kubeadm init --skip-preflight-checks

Create cloud-config file and change args for kubelet

  • Change your credentials and run this command
cat << EOF > /etc/kubernetes/cloud-config
[Global]
auth-url=https://<your keystone api>:5000/v3
domain-name=Default
tenant-name=<your tenant name>
username=<your username>
password=<your password>
region=<your region>
EOF
  • Note: currently we don’t set any config about Loadbalancer, because loadbalancer has not deployed yet in cloud v2 (soon)

  • Change args for kubelet

sed -i -E 's/(.*)KUBELET_KUBECONFIG_ARGS=(.*)$/\1KUBELET_KUBECONFIG_ARGS=--cloud-provider=openstack --cloud-config=\/etc\/kubernetes\/cloud-config \2/' /etc/systemd/system/kubelet.service.d/10-kubeadm.conf
  • Restart kubelet
systemctl daemon-reload
systemctl restart kubelet

Create Kubernetes cluster

  • Generate kubeadm config file
kind: MasterConfiguration
apiVersion: kubeadm.k8s.io/v1alpha1
cloudProvider: openstack
kubernetesVersion: 1.9.3
unifiedControlPlaneImage: "gcr.io/google_containers/hyperkube-amd64:v1.9.3"
networking:
  podSubnet: 192.168.0.0/16

Notes: change these options if you know - cloudProvider: openstack: use this config for Openstack Cloud - podSubnet: 192.168.0.0/16: use this for calico network

  • Init cluster
kubeadm init --config /path/to/kubeadm-config.yml
  • After kube-apiserver and kube-controller-manager running you can setup network plugin, in here we are using calico plugin, run following command
kubectl apply -f https://docs.projectcalico.org/v3.0/getting-started/kubernetes/installation/hosted/kubeadm/1.7/calico.yaml
  • After kube-dns is running, you can join worker node to master node

Note: You have to copy cloud-config to other nodes and change kubelet args like master node

Launch application use cinder volume

  • Firstly, we have to create storageclass
cat << EOF > cinder-storage-class.yml
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: hdd1
provisioner: kubernetes.io/cinder
parameters:
  type: HDD1 # change for your cloud volume type
  availability: nova
EOF
  • Create storage class
kubectl create -f cinder-storage-class.yml
  • Use helm to install grafana, change storageClass in values.yml file

After install

[email protected]:~/charts/stable/grafana# helm ls
NAME    REVISION        UPDATED                         STATUS          CHART           NAMESPACE
grafana 1               Thu Feb 22 20:18:07 2018        DEPLOYED        grafana-0.7.0   default  
  • Show pvc, pv
[email protected]:~/charts/stable/grafana# kubectl get pvc,pv
NAME                  STATUS    VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
pvc/grafana-grafana   Bound     pvc-d25639fa-17d2-11e8-bfef-fa163e679d2b   1Gi        RWO            hdd1           36m

NAME                                          CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS    CLAIM                     STORAGECLASS   REASON    AGE
pv/pvc-d25639fa-17d2-11e8-bfef-fa163e679d2b   1Gi        RWO            Delete           Bound     default/grafana-grafana   hdd1                     36m
Reference
  • https://sapham.net/install-kubernetes-using-kubeadm-with-calico-network-2/
  • https://kubernetes.io/docs/setup/independent/create-cluster-kubeadm/
  • https://kubernetes.io/docs/setup/independent/install-kubeadm/
  • https://kubernetes.io/docs/concepts/storage/storage-classes/#openstack-cinder
  • https://kubernetes.io/docs/concepts/cluster-administration/cloud-providers/#openstack