"Failed to start vc-ws1a-broker services. Error: A system error occurred." when patching vCenter

Products

• VMware vCenter Server

Issue/Introduction

When attempting to patch vCenter Server 8.x, the process fails at 90% because the vc-ws1a-broker service fails to start. This issue occurs after running the vCenter 9 RDU upgrade precheck if the upgrade process is not completed.

In /var/log/vmware/vmon/vmon.log 

<vc-ws1a-broker> Service pre-start command's stderr:     raise Exception('Command failed: stdout %r stderr %r' % (text, err))
level=error msg="runc run failed: unable to start container process: error during container init: error setting cgroup config for procHooks process: bpf_prog_query(BPF_CGROUP_DEVICE) failed: invalid argument" <vc-ws1a-broker> Service pre-start        command failed with exit code 1.
<vc-ws1a-broker> Service pre-start command failed with exit code 1

Environment

• vCenter 8.x
• vCenter 9.x

Cause

After running the vCenter 9 RDU upgrade precheck and not moving forward with the upgrade for any reason when rebooting the vCenter 8 appliance, the Workspace ONE broker service will fail to start. 

The 9.1 precheck script modifies the vCenter 8 container cgroup configuration in the /boot/systemd.cfg file, enabling a unified cgroup hierarchy that is incompatible with the vCenter 8.x container runtime.

Resolution

Revert the cgroup configuration to legacy mode:

1. Take a snapshot of the vCenter Server prior to making any changes
2. Log in to the vCenter Server Appliance via SSH as the root user
3. Edit the /boot/systemd.cfg file
4. Change the value systemd.unified_cgroup_hierarchy=yes to systemd.legacy_systemd_cgroup_controller=yes
5. Save the file and perform a full reboot of the vCenter appliance
6. Verify that all services including vc-ws1a-broker are running using the command: service-control --status --all

Additional Information

For further assistance, contact Broadcom Support

• Contact Broadcom Support
• Creating and Managing Broadcom Support Cases