rnb-controller is not processing any backup, backupschedule or restoration

Products

• VMware Integrated OpenStack

Issue/Introduction

Symptoms:

• Backup and/or restoration process cannot be processed.
  • For example, after running "viocli restore deployment", this process seems to be stuck after displaying "restore initiated".
• Failed backup
• In rnb-controller.log, the last line shows resync time is 15s and this service seems to be stuck.

oslog deploy/rnb-controller rnb-controller

• rnb-controller.log is empty
  oslog deploy/rnb-controller rnb-controller

Environment

7.x

Cause

This issue can happen when vio-manager and controller where rnb-controller pod runs are rebooted around the same time due to kube-apiserver not running.

Resolution

This issue can be resolved by restarting rnb-controller via the following command.

osctl rollout restart deploy rnb-controller

Check the rnb-controller logs and make sure it is logging

oslog deploy/rnb-controller rnb-controller

Or you can run these commands:

#osctl get pods -n openstack | grep rnb-controller

#osctl delete pod rnb-controller-<name from above>

The pod will delete and recreate.  You can run this to watch it recreate.

#osctl get pods -n openstack | grep rnb-controller