vCenter update fails to complete with error ""Exception occurred in postInstallHook for B2B-pathing."

Products

• VMware vCenter Server

Issue/Introduction

• vCenter Update fail @ 80% with errors:
  "Exception occurred in postInstallHook for B2B-pathing. Please check the logs for more details. Take corrective action and then resume."
• In var/log/vmware/applmgmt/PatchRunner.log file you can see lines similar to:

<timestamp> wcp:Patch ERROR wcp Failed to apply patch %s! Error: %s.
<timestamp> wcp:Patch ERROR wcp Not all patches were applied. Latest applied patch is 1
<timestamp>  wcp:Patch ERROR vmware_b2b.patching.executor.hook_executor Patch hook 'wcp:Patch' failed.
patch_errors.UserError: Failed to apply patch roles_groups_users! Error: Role WorkloadStorageManagement (id: 1004) not found in VC..

OR

patch_errors.UserError: Failed to apply patch roles_groups_users! Error: Role NSOperatorController (id: 1023) not found in VC.

 

Environment

vCenter Server 8.x

Cause

The role names and roleids are present but the role id have an invalid value.

The following Roles are built for use by the WCP service on vCenter:

Role name in vmdir	Roleid	Role Name in vSphere Web client
WorkloadStorageManagement	1004	Workload Storage Manager
vSphereKubernetesManager	1005	vSphere Kubernetes Manager
SupervisorServiceCluster	1007	Supervisor Cluster Operator
SupervisorServiceRootFolder	1008	SupervisorService RootFolder Operator
SupervisorServiceGlobal	1009	Supervisor Operator
VMOperatorController	1021	VMOperator Controller
VMOperatorControllerGlobal	1022	VMOperator Controller Manager
VMServicesAdministrator	0 (Authz  generated)	VM Services Administrator
NsxAuditor	0 (Authz  generated)	NSX Auditor
NsxViAdministrator	0 (Authz  generated)	NSX VI Administrator
NsxAdministrator	0 (Authz  generated)	NSX Administrator
NSOperatorController	1023	NSOperatorController

Resolution

Note: Altering vCenter VMDIR database could be destructive. Please ensure to have valid backups/snapshots of vCenter (step 1) before applying the resolution steps (contact Broadcom support if needed).

 

• The script from the Manually Synchronize WCP Roles may report that all the roles exist. but service start fails. 
• Gather the lidf export by running: (replace <sso_password> by your administrator password)

ldapsearch -o ldif-wrap=no -LLL -h localhost -b "cn=RoleModel,cn=VmwAuthz,cn=Services,dc=vsphere,dc=local" -s sub -D "cn=Administrator,cn=Users,dc=vsphere,dc=local" -w '<sso_password>' > /var/tmp/role.ldif 

• • From the vmon logs, search for role name and id that the wcp service is failing to start with from the ldif output.

  • Correct roleid will look like the following  

    dn: cn=1005,cn=RoleModel,cn=VmwAuthz,cn=services,dc=vsphere,dc=local
cn: 1005
objectClass: top
objectClass: vmwAuthzRole
nTSecurityDescriptor:: ########
vmwAuthzRolePrivilegeId: System.Anonymous
vmwAuthzRolePrivilegeId: System.Read
vmwAuthzRolePrivilegeId: System.View
vmwAuthzRolePrivilegeId: VirtualMachine.Config.AddRemoveDevice
vmwAuthzRolePrivilegeId: Cryptographer.Clone
vmwAuthzRolePrivilegeId: VirtualMachine.Config.EditDevice
vmwAuthzRolePrivilegeId: Cryptographer.Migrate
vmwAuthzRolePrivilegeId: Cryptographer.Encrypt
vmwAuthzRolePrivilegeId: VirtualMachine.Interact.DeviceConnection
vmwAuthzRolePrivilegeId: VirtualMachine.Inventory.Delete
vmwAuthzRolePrivilegeId: Cryptographer.AddDisk
vmwAuthzRolePrivilegeId: VirtualMachine.Inventory.Create
vmwAuthzRolePrivilegeId: VirtualMachine.Config.AddNewDisk
vmwAuthzRolePrivilegeId: Cryptographer.Recrypt
vmwAuthzRolePrivilegeId: VirtualMachine.Interact.PowerOn
vmwAuthzRolePrivilegeId: InventoryService.Tagging.ObjectAttachable
vmwAuthzRolePrivilegeId: Datastore.AllocateSpace
vmwAuthzRolePrivilegeId: Cryptographer.EncryptNew
vmwAuthzRolePrivilegeId: VirtualMachine.Interact.PowerOff
vmwAuthzRolePrivilegeId: Resource.AssignVMToPool
vmwAuthzRoleVersion: 1
vmwAuthzRoleName: vSphereKubernetesManager
vmwAuthzRoleDescription: This role entitles the vSphere Kubernetes Manager to perform lifecycle management of vSphere Pods, including vSphere Pod scheduling, creation, deletion and attaching of devices.

  • While an invalid role will look like: 

dn: cn=<id_not_1005>,cn=RoleModel,cn=VmwAuthz,cn=services,dc=vsphere,dc=local
nTSecurityDescriptor:: ########
objectClass: top
objectClass: vmwAuthzRole
cn: <id_not_1005>
vmwAuthzRolePrivilegeId: System.Anonymous
vmwAuthzRolePrivilegeId: System.Read
vmwAuthzRolePrivilegeId: System.View
vmwAuthzRolePrivilegeId: VirtualMachine.Config.AddRemoveDevice
vmwAuthzRolePrivilegeId: VirtualMachine.Inventory.Delete
vmwAuthzRolePrivilegeId: VirtualMachine.Interact.PowerOn
vmwAuthzRolePrivilegeId: ContentLibrary.ManageClusterRegistryResource
vmwAuthzRolePrivilegeId: Datastore.AllocateSpace
vmwAuthzRolePrivilegeId: VirtualMachine.Interact.PowerOff
vmwAuthzRolePrivilegeId: VirtualMachine.Inventory.Create
vmwAuthzRolePrivilegeId: Resource.AssignVMToPool
vmwAuthzRolePrivilegeId: VirtualMachine.Config.AddNewDisk
vmwAuthzRoleVersion: 2
vmwAuthzRoleName: vSphereKubernetesManager
vmwAuthzRoleDescription:: IA==

 

• To fix this:

 

• • Step1: Take a snapshot of the VC

https://knowledge.broadcom.com/external/article/313886/vmware-vcenter-in-enhanced-linked-mode-p.html

Use of offline snapshots in ELM deployments is very strongly recommended for a safe rollback point. This means all appliances should be gracefully shut down, and snapshots need to be taken while the VCSAs are in powered off state (at the same time).

If any change must be reverted, restore all of the nodes in the ELM deployment to this offline/consistent snapshot state.

• • Step2: Stop vpxd

#service-control --stop vpxd

• • Step3: Delete the role with the invalid role by running (replace <id_not_1005> by the id from your ldif export)
    ldapdelete -H ldap://localhost -D "cn=administrator,cn=users,dc=vsphere,dc=local" -w "<password>" "cn=<id_not_1005>,cn=RoleModel,cn=VmwAuthz,cn=services,dc=vsphere,dc=local"

• • Step4: Create a /usr/lib/vmware-wcp/py-modules/roles.py file and import config using the following command (ensure you copy the entire text from cat to the last EOF line):

    # cat <<EOF >>/usr/lib/vmware-wcp/py-modules/roles.py

    import featureState
    from roles_groups_users_utils import Authz

    if __name__ == "__main__":
        authz_patch = Authz()

    authz_patch.ensure_roles_exist('/usr/lib/vmware-wcp/roles.xml')
    authz_patch.ensure_roles_exist('/usr/lib/vmware-wcp/nsop-roles.xml')

    EOF

• • Step5: Set the Python path with the following command:

    # export PYTHONPATH="${PYTHONPATH}:/usr/lib/vmware/site-packages"

  • Step6: Run the python script to recreate roles using the following command:
    # python /usr/lib/vmware-wcp/py-modules/roles.py
     
    Example Output:

    # python /usr/lib/vmware-wcp/py-modules/roles.py
2025-11-21T13:29:46.083Z  Role id 1004, name WorkloadStorageManagement - already exists
2025-11-21T13:29:46.086Z  Role id 1005, name vSphereKubernetesManager - added
2025-11-21T13:29:46.089Z  Role id 1007, name SupervisorServiceCluster - already exists
2025-11-21T13:29:46.091Z  Role id 1008, name SupervisorServiceRootFolder - already exists
2025-11-21T13:29:46.094Z  Role id 1009, name SupervisorServiceGlobal - already exists
2025-11-21T13:29:46.108Z  Role id 1021, name VMOperatorController - already exists
2025-11-21T13:29:46.114Z  Role id 1022, name VMOperatorControllerGlobal - already exists
2025-11-21T13:29:46.118Z  Role id 0, name VMServicesAdministrator - already exists
2025-11-21T13:29:46.125Z  Role id 0, name NsxAuditor - already exists
2025-11-21T13:29:46.130Z  Role id 0, name NsxViAdministrator - already exists
2025-11-21T13:29:46.138Z  Role id 0, name NsxAdministrator - already exists
2025-11-21T13:29:46.157Z  Role id 1023, name NSOperatorController - added
    
    

                You will now see that role has been added. 

• • Step7: Start vpxd

#service-control --start vpxd

• • Step8:Start the WCP service

#service-control --restart wcp

• • Step9: Update vCenter