SDDC Manager upgrade stalls with NSX Edge status UNDEFINED

Products

• VMware SDDC Manager / VCF Installer

Issue/Introduction

During a VMware Cloud Foundation (VCF) upgrade, the SDDC Manager UI shows the upgrade task as running or failed, while NSX Edge nodes report a status of `UNDEFINED` or `UNKNOWN`. vCenter may show that host upgrades are complete, but the SDDC Manager inventory remains out of sync.

NSX Edge Transport Nodes show status `UNDEFINED` : `UNKNOWN`.
SDDC Manager upgrade tasks stall at 60% or remain in an `INPROGRESS` state indefinitely

Environment

   VMware Cloud Foundation 5.x
   SDDC Manager
   VMware NSX 4.x

Cause

Stale deployment locks in the SDDC Manager database or a mismatch between the `lcm` service state and the actual resource status prevent the upgrade from progressing.

Resolution

1.  SSH to the SDDC Manager appliance as the `vcf` user and elevate to `root` (`su -`).
2.  Verify current deployment locks:
        curl localhost/lockhost/locks | jq
    
3.  Clear the locks in the platform database:
        psql -h localhost -U postgres -d platform -c "truncate lock;"
    
4.  Identify the stalled upgrade element:
        psql -h localhost -U postgres -d platform -c "SELECT id, upgrade_id, upgrade_status FROM lcm.upgrade_element WHERE upgrade_status='INPROGRESS';"
    
5.  Update the status of the stalled element to success:
        psql -h localhost -U postgres -d platform -c "UPDATE lcm.upgrade_element SET upgrade_status='COMPLETED_WITH_SUCCESS' WHERE id='####';"
    
    *Note: Replace #### with the ID identified in step 4.*

6.  Restart the LCM service:
        systemctl restart lcm
    
   

7.  If the issue persists, contact Broadcom Support

 

For defects and enhancements, subscribe to this article to be updated on fix status  Subscribe to a Broadcom knowledge article by article or product

Additional Information

For information on general NSX upgrade limitations, see Operational Impact of the NSX Upgrade