vCenter Server upgrade fails at VCENTER_UPGRADE_PREPARATION with KeyError: 'WorkloadStorageManagement'

Products

• VMware SDDC Manager / VCF Installer
• VMware vCenter Server 8.0

Issue/Introduction

When upgrading vCenter Server (e.g., from 8.0U3d to 8.0U3g) within a VMware Cloud Foundation (VCF) environment using SDDC Manager, the process fails during the preparation stage.

Symptoms:

• SDDC Manager UI displays:
  vCenter Server upgrade fails at VCENTER_UPGRADE_PREPARATION stage.
  
  
• The following error was recorded in lcm.log on SDDC Manager :
  Unable to start all services during the upgrade
  
  
• The vmon.log on the target vCenter Server contains the following Python Traceback:
  File "/usr/lib/vmware-wcp/py-modules/wcpconfigure.py", line 541, in register_with_ls
...
File "/usr/lib/vmware/site-packages/cis/cisreglib.py", line 728, in create_access_control
role_ids.append(role_objs_dict[rolename])
KeyError: 'WorkloadStorageManagement'
  
  
• The authz-data.json in the support bundle shows the role name is localized in Japanese ("ワークロード ストレージ マネージャ") :
  "name": "\u30ef\u30fc\u30af\u30ed\u30fc\u30c9 \u30b9\u30c8\u30ec\u30fc\u30b8 \u30de\u30cd\u30fc\u30b8\u30e3"
  
  

Environment

SDDC Manager: 5.x
vCenter Server: 8.0.x

Cause

• The upgrade/patching script for the Workload Control Plane (WCP) service expects the specific English string WorkloadStorageManagement to register permissions.
• If this role has been renamed or localized to ワークロード ストレージ マネージャ, the cisreglib.py library fails to find the matching ID in the internal dictionary, resulting in a KeyError and preventing the wcp service from starting.

Resolution

1. Check the settings for users, groups, etc., currently assigned to the role "ワークロード ストレージ マネージャ".

Check the settings with the following operations.
Please take screenshots just in case.

vSphere Client -> Administration -> Access Control -> Roles -> "ワークロード ストレージ マネージャ" -> [Usage]
vSphere Client -> Administration -> Access Control -> Global Permissions -> <User/Group name confirmed above> -> Edit -> Cancel

2. Delete the role "ワークロード ストレージ マネージャ"

vSphere Client -> Administration -> Access Control -> Roles -> "ワークロード ストレージ マネージャ" -> Delete

3. Add the role "ワークロード ストレージ マネージャ" by following the steps in KB 319396 Resolution.

4. Revert to the settings confirmed in 1.

5. Attempt the vCenter Server upgrade from SDDC Manager.

Additional Information

Manually Synchronize WCP Roles
https://knowledge.broadcom.com/external/article?articleNumber=319396

Japanese version of this KB : https://knowledge.broadcom.com/external/article/431108