vCenter Patching fails at 80% Error: Exception occurred in postHookInstall due to role not found in VC

Products

• VMware Cloud Foundation
• VMware vCenter Server

Issue/Introduction

Check and fix if vmAuthzRoleName exists or has been modified

Symptoms:

• Patching vCenter 7.0.3 fails 

• Error in /var/log/vmware/applmgmt/PatchRunner.log

[YYYY-MM-DDTHH:MM:SS] wcp:Patch ERROR wcp Failed to apply patch %s! Error: %s. 
[YYYY-MM-DDTHH:MM:SS] wcp:Patch ERROR wcp Not all patches were applied. Latest applied patch is 1 
[YYYY-MM-DDTHH:MM:SS] wcp:Patch ERROR vmware_b2b.patching.executor.hook_executor Patch hook 'wcp:Patch' failed.
Traceback (most recent call last):
  File "/storage/updatemgr/software-updateqvoymy0v/stage/scripts/patches/py/vmware_b2b/patching/executor/hook_executor.py", line 74, in executeHook
    executionResult = systemExtension(args)
  File "/storage/updatemgr/software-updateqvoymy0v/stage/scripts/patches/libs/sdk/extensions.py", line 106, in __call__
    result = self.extension(*args)
  File "/storage/updatemgr/software-updateqvoymy0v/stage/scripts/patches/libs/sdk/extensions.py", line 123, in _func
    return func(*args)
  File "/storage/core/software-update/updates/7.0.3.01000/scripts/patches/payload/components-script/wcp/__init__.py", line 213, in doPatching
    doIncrementalPatching(current_version)
  File "/storage/core/software-update/updates/7.0.3.01000/scripts/patches/payload/components-script/wcp/__init__.py", line 340, in doIncrementalPatching
    raise user_error
patch_errors.UserError: Failed to apply patch roles_groups_users! Error: Role VMOperatorController not found in VC.. 
[YYYY-MM-DDTHH:MM:SS] ERROR vmware_b2b.patching.phases.patcher Patch hook Patch got ComponentWrapperError.
Traceback (most recent call last):
  File "/storage/updatemgr/software-updateqvoymy0v/stage/scripts/patches/py/vmware_b2b/patching/phases/patcher.py", line 203, in patch
    _patchComponents(ctx, userData, statusAggregator.reportingQueue)
  File "/storage/updatemgr/software-updateqvoymy0v/stage/scripts/patches/py/vmware_b2b/patching/phases/patcher.py", line 85, in _patchComponents
    executeComponentHook(Hook.Patch, ctx, c, userData, reportingQueue)
  File "/storage/updatemgr/software-updateqvoymy0v/stage/scripts/patches/py/vmware_b2b/patching/executor/execution_facade.py", line 98, in executeComponentHook
    reportQueue, identifier, expectedResultType)
  File "/storage/updatemgr/software-updateqvoymy0v/stage/scripts/patches/py/vmware_b2b/patching/executor/execution_facade.py", line 53, in executeHook
    result = executor.executeHook(scriptFile, hook, args, reportQueue, reportIdentifier)
  File "/storage/updatemgr/software-updateqvoymy0v/stage/scripts/patches/py/vmware_b2b/patching/executor/hook_executor_process.py", line 119, in executeHook
    raise ex
patch_errors.ComponentError 
[YYYY-MM-DDTHH:MM:SS] WARNING root stopping status aggregation... 
[YYYY-MM-DDTHH:MM:SS] ERROR __main__ Patch vCSA failed

Environment

VMware vCenter Server 7.0.x
VMware Cloud Foundation 4.4.x

Cause

This could occur if the environment has a history of role related issues or default roles have been manually modified

Resolution

• Take powered-off snapshots of vCenter appliance
• Check the roles by accessing Jxplorer Services>vmAuthz>RoleModel as displayed in the snippet below

If JXplorer isn't available in the customer environment, export roles ldif using following command to view it offline

 

ldapsearch -o ldif-wrap=no -LLL -h localhost -b "cn=RoleModel,cn=VmwAuthz,cn=Services,dc=vsphere,dc=local" -s sub -D "cn=Administrator,cn=Users,dc=vsphere,dc=local" -w 'Password' > /var/tmp/role.ldif

 

• Rename the role to VMOperatorController

In JXplorer 1021> TableEditor and Submit
or
Using command line

ldapmodify -x -h localhost -p 389 -D "cn=administrator,cn=users,dc=vsphere,dc=local" -W <<EOF
dn: cn=1021,cn=RoleModel,cn=VmwAuthz,cn=Services,dc=vsphere,dc=local
changetype: modify
replace: vmwAuthzRoleName
vmwAuthzRoleName: VMOperatorController
EOF

• Restart services

service-control --stop --all && service-control --start --all