VxRail upgrade from SDDC Manager failed in vLCM enabled cluster with an error "The error message: 1 error(s) meets. Details: 1.Health Check for... failed."

Products

• VMware vCenter Server
• VMware SDDC Manager
• VMware Cloud Foundation
• VMware NSX

Issue/Introduction

• Upgrade workflow reports the following error:
  
  VxRail upgrade failed in vLCM enabled cluster with an error "Failure occurred while running an upgrade for bundle: 
<bundle_id>. The error message: 1 error(s) meets. Details: 1.Health Check for '<cluster_name>' failed."
  
  
• SDDC Manager logs shows the following errors:
  
  lcm.log
xxxx-xx-xxTxx:xx:xx.xxx+0000 ERROR [vcf_lcm,xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx,xxxx] [c.v.evo.sddc.lcm.model.task.SubTask,http-nio-127.0.0.1-7400-exec-4] Upgrade error occured: [vxrail updateUpgradeProgress]
VxRail Upgrade returned status as Failure : Failure occurred while running an upgrade for bundle: <bundle_id>. The error message: 1 error(s) meets. Details: 1.Health Check for 'cluster_name' failed.
Reference token XXXXXX
  
  
• vCenter logs shows the following errors:
  
  vum-server.log
xxxx-xx-xxTxx:xx:xx.xxxz info vmware-vum-server[xxxxxx] [Originator@6876 sub=IntegrityAuthzServiceHelper opID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx] [AuthzServiceHelper 69] 
Entity Priv Result: PrivId:VcIntegrity.lifecycleSoftwareSpecification.Read, PrivValue:false
xxxx-xx-xxTxx:xx:xx.xxxz info vmware-vum-server[xxxxxx] [Originator@6876 sub=IntegrityAuthzServiceHelper opID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx] [AuthzServiceHelper 186] 
User: [email protected] does not have Privilage: VcIntegrity.lifecycleSoftwareSpecification.Read on entity:domain-cxx
  
  
• Looking at the vCenter Global permissions we are seeing the nsx service account permissions that have their domain mapping showing as NULL
  
  

 

Environment

VMware Cloud Foundation
VMware vCenter Server
VMware SDDC Manager
VMware NSX

Resolution

To resolve the issue login to vSphere Client under Administration/Global permissions delete and and re-apply the permission and role to the specific nsx service account.

After the permissions re-applied we should see the domain mapping pointing correctly to the domain SSO.

Additional Information

Using vCenter Server Global Permissions