VIO Octavia loadbalancer stuck in a PENDING_UPDATE status

Products

• VMware
• VMware Integrated OpenStack

Issue/Introduction

Currently, there is no built-in mechanism to change the load balancer into an error state for easy removal through the User Interface.

Symptoms:
Load Balancer can not be deleted
Load Balancer is in a PENDING_UPDATE status.

The error message:
OctaviaClientException: Invalid state PENDING_UPDATE of loadbalancer resource e9317a9e-f286-4471-9302-f871fda2d20d (HTTP 409) (Request-ID: req-8a1efe7a-84cd-4d64-bea0-17d0de684d6a)

Environment

VMware Integrated Openstack 7.x

Resolution

This is a known issue affecting VMware Integrated Openstack VIO7.x

Workaround:
Note: This procedure modifies the database. Ensure to take a backup of the database before proceeding.   VIO backup and recovery best practices (2150107)

1. Please make sure no object exists for these LBs in NSX (just search by their openstack id using NSX UI). If any object present in NSX, please clean them up as well.
2.  Log in to the LCM node and log in to a mariadb pod:

kubectl -n openstack exec -it mariadb-server-0 bash

3. Find the loadbalancer id in ocatavia db and update its status.

Example:
root@mariadb-server-0:/# mysql --defaults-file=/etc/mysql/admin_user.cnf --host=localhost octavia -e 'select * from load_balancer';

root@mariadb-server-0:/# mysql --defaults-file=/etc/mysql/admin_user.cnf --host=localhost octavia -e'update load_balancer set provisioning_status='ERROR' where id = "e9317a9e-f286-4471-9302-f871fd";'

4. Delete the loadbalancer in error state using openstack CLI:

openstack loadbalancer list --operating-status=ERROR
openstack loadbalancer delete <load balancer ID>

Additional Information

Impact/Risks:
Note: This procedure modifies the database. Ensure to take a backup of the database before proceeding