mariadb-server-0 pod is in a CrashLoopBackoff state due to an empty PID file

Products

• VMware Integrated OpenStack

Issue/Introduction

• Unable to restart VIO services.
• Startup process of the MariaDB server fails within the Kubernetes pod.

Cause

The PID file of the MariaDB server process, is empty causing Kubernetes to repeatedly restart the pod.

Resolution

Delete mariadb-server-0.pid file and restart mariadb-server-0 pod

1. Login to a VIO controller where  mariadb-server-0 pod is running and locate 'mariadb-server-0.pid' file in one of the PVC volume directory.
   

   Example
   root@controller-lbzfzskd88 [ ~ ]# find /var/lib/kubelet/pods/ -print | grep mariadb-server-0.pid
   /var/lib/kubelet/pods/92ec3ff6-7343-4e03-b157-9b66c8a7f24f/volumes/kubernetes.io~vsphere-volume/pvc-44fb56f1-d0f8-4056-8a6c-5974f201ee79/mariadb-server-0.pid

   root@controller-lbzfzskd88 [ ~ ]# cd /var/lib/kubelet/pods/92ec3ff6-7343-4e03-b157-9b66c8a7f24f/volumes/kubernetes.io~vsphere-volume/pvc-44fb56f1-d0f8-4056-8a6c-5974f201ee79/
   
   

2. Remove mariadb-server-x.pid file
   rm mariadb-server-0.pid


3. Restart mariadb-server-0 pod on VIO manager.
   osdel pod mariadb-server-0 --force  --grace-period=0