The NSX upgrade tab never finishes loading

Products

• VMware NSX

Issue/Introduction

• Unable to perform any tasks from System -> Upgrade in NSX UI, the page fails to load properly and displays only a spinning circle indefinitely.
• NSX prepared clusters are vSphere Life Cycle Manager (vLCM) enabled clusters.

• You see log entries similar to the following in the NSX manager log /var/log/proton/localhost_access_log.txt log:

  127.0.0.1 - - "GET /upgrade-coordinator/api/v1/upgrade/status-summary HTTP/1.1" 400 320

OR

127.0.0.1 - - "GET /upgrade-coordinator/api/v1/upgrade/status-summary HTTP/1.1" 500 272

• You see log entries similar to the following in the NSX manager log /var/log/upgrade-coordinator/upgrade-coordinator.log
  

  ERROR http-nio-127.0.0.1-7442-exec-3 ManagedESXUpgradeManager 3155 SYSTEM [nsx@6876 comp="nsx-manager" errorCode="MP30973" level="ERROR" subcomp="upgrade-coordinator"] [HUC] Cannot move <ESXi host> upgrade unit to Group for <ClusterName> group as vLCM is enabled on it.

  INFO http-nio-127.0.0.1-7442-exec-3 FacadeInterceptorHelperImpl 3155 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="upgrade-coordinator"] Caught error in facade interceptor com.vmware.nsx.management.upgrade.exceptions.InvalidPlanException: null

INFO http-nio-127.0.0.1-7442-exec-1 NsxBaseRestController 3109 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="upgrade-coordinator"] Error in API /upgrade-coordinator/api/v1/upgrade/status-summary caused by exception com.vmware.nsx.management.upgrade.exceptions.InvalidPlanException:  {"moduleName":"upgrade-coordinator","errorCode":30973,"errorMessage":"Cannot move '<ESXi host>' upgrade unit to Group for 'ClusterName' group as vLCM is enabled on it."}

OR

ERROR localhost-startStop-1 UpgradePlanSynchronizationServiceImpl 5471 SYSTEM [nsx@6876 comp="nsx-manager" errorCode="MP30080" level="ERROR" subcomp="upgrade-coordinator"] Sync Plan Failed. Ignoring this exception.
com.vmware.nsx.management.upgrade.exceptions.InvalidPlanException: null

INFO http-nio-127.0.0.1-7442-exec-3 NsxBaseRestController 5467 SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="upgrade-coordinator"] Error in API /upgrade-coordinator/api/v1/upgrade/status-summary caused by exception com.vmware.nsx.management.upgrade.exceptions.InvalidPlanException:  {"moduleName":"upgrade-coordinator","errorCode":30479,"errorMessage":"Reboot less upgrade cannot be disabled for vSphere Lifecycle Managed cluster '########-####-####-####-############:domain-c##'."}

• Please note there is another KB article which identifies an issue that presents with same symptom: The NSX upgrade tab never finishes loading and displays a spinning circle indefinitely. but the cause and resolution is different and can only be ascertained from findings in the upgrade-coordinator.log file.

Environment

VMware NSX-T Data Center
VMware NSX

Cause

This issue can occur with vLCM enabled cluster(s), when the MUB is uploaded, the upgrade plan for the hosts is created based on existing hosts, but when a host is added/removed on an NSX prepared vLCM cluster, this differs from the plan. 

This issue can also occur if one of the Host Clusters was converted from VUM to vLCM mid-upgrade i.e. if Upgrade has been started with UC upgraded but the Manager Appliances have not been upgraded yet.

Resolution

This issue is resolved in VMware NSX 4.1.1, available at Broadcom downloads.

If you are having difficulty finding and downloading software, please review the Download Broadcom products and software KB.

 

To workaround this issue, issue the following command from the NSX manager CLI (while logged in as root):

curl -ku 'admin' -X POST 'https://localhost/api/v1/upgrade/plan?action=reset&component_type=HOST'

 

Please note that the above command needs to be run on all Manager nodes in the Cluster