AVI Controller upgrade stuck at 93% while UI shows "In Progress"

Products

• VMware Avi Load Balancer

Issue/Introduction

• The AVI Controller upgrade to 22.1.5 stuck at 93%
• The controller UI shows "In Progress" and is stuck at 93% Complete.
  
  
• From the CLI "show upgrade status detail" output, it shows 30 tasks completed out of 32.
  
  
• show upgrade status
+---------------------------+--------+---------------+------------------------------------------+-----------+-----------------------------+-------+--------+
| Name                      | Tenant | Cloud         | State                                    | Operation | Image                       | Patch | Reason |
+---------------------------+--------+---------------+------------------------------------------+-----------+-----------------------------+-------+--------+
| cluster-#-#               | admin  | -             | UPGRADE_FSM_CONTROLLER_COMPLETED : (93)% | UPGRADE   | 22.1.5-9093-20231010.145553 | -     | -      |
| Default-Group             | admin  | Default-Cloud | UPGRADE_FSM_COMPLETED                    | UPGRADE   | 22.1.5-9093-20231010.145553 | -     | -      |
| ##-#####-###              | admin  | ###-####-#### | UPGRADE_FSM_COMPLETED                    | UPGRADE   | 22.1.5-9093-20231010.145553 | -     | -      |

Environment

• VMware AVI Load Balancer
  
  • 22.1.x

Cause

• The upgrade process has successfully completed in the backend.
• The database cache (DbCache) is not properly refreshing after the upgrade, causing the CLI/UI to show outdated data despite the successful backend process.

Resolution

• The issue has been fixed on the following versions:
  
  • 30.1.1
  • 30.2.1
    
    
• Workaround:
  • SSH to the controller leader node and run the command below to restart the service.
  • systemctl restart process-supervisor.service
    
    
• Note: UI will be unavailable after restarting the service for some time.