Using Ceph as storage for Kubernetes clusters

Wouldn’t be a nice feature to use Rados Block Devices for Kubernetes persistent storage?
Most of the steps used to setup a basic Kubernetes cluster are presented here and will detail only the Ceph integration part.

Presuming that Kubernetes cluster is up and running jumping to the next steps:

Creating the rbd volume

On cephadmin node:

rbd create kube_ceph -s 10240 --image-feature layering

rbd map kube_ceph

mkfs.xfs /dev/rbd0

rbd unmap kube_ceph

 

Preparing Kubernetes to embrace Ceph

 

On any ceph node:

ceph auth get-key client.admin

echo "AQDT5ppYCuM3ChAAXOkw8h4ii7qsnybdXV9D1g==" | base64 > secret.txt

 

On Kubernetes master node:

vi ceph-secret.yml

apiVersion: v1
kind: Secret
metadata:
   name: ceph-secret
data:
   key: < cat secret.txt here >

 

vi ceph-pv.yml

apiVersion: v1
kind: PersistentVolume
metadata: 
   name: ceph-pv
spec:
   capacity:
      storage: 10Gi
   accessModes:
      – ReadWriteMany
   rbd: 
      monitors: 
         – knode01:6789
         – knode02:6789
         – knode03:6789
     pool: rbd
        image: kube_ceph
        user: admin
        secretRef: 
           name: ceph-secret
        fsType: xfs
        readOnly: false

 

vi ceph-pv-claim.yml

kind: PersistentVolumeClaim
apiVersion: v1
metadata:
   name: ceph-claim
spec:
   accessModes:
      – ReadWriteMany
   resources:
      requests:
      storage: 10Gi

 

kubectl create -f ceph-secret.yml

kubectl get secret

You should receive a similar output:

[root@master ~]# kubectl get secret
NAME TYPE DATA AGE
ceph-secret Opaque 1 1d
default-token-cgstk kubernetes.io/service-account-token 3 2d

 

kubectl create -f ceph-pv.yml

kubectl create -f ceph-pv-claim.yml

 

Now let’s create a MySQL pod with Ceph Persistent Storage

vi myqsl-pod.yml

apiVersion: v1
kind: Pod
metadata:
   name: ceph-mysql
spec:
   containers:
      – name: ceph-mysql
         image: tutum/mysql
      ports:
         – name: mysql-db
            containerPort: 3306
      volumeMounts:
         – name: mysql-pv
           mountPath: /var/lib/mysql
   volumes:
      – name: mysql-pv
         persistentVolumeClaim:
            claimName: ceph-claim

 

kubectl create -f mysql-pod.yml

 

And now you can enjoy your new MySQL container that has Ceph backed storage.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s