vCenter Server 8.0 patching fails with "An error occurred while invoking external command : certool" during wcp:Patch phase

Products

• VMware vCenter Server

Issue/Introduction

• Upgrading or patching vCenter Server 8.0 fails.
• The update fails specifically at the roles_groups_users stage while processing the wcp solution user.
• The appliance patching logs (located at /var/log/vmware/applmgmt/patchrunner.log) contain the following exception stack trace:

[YYYY-MM-DDTHH:MM:SS] wcp:Patch INFO root Removing certificate for user "wcp" from the filesystem
[YYYY-MM-DDTHH:MM:SS] wcp:Patch ERROR wcp Failed to apply patch %s! Error: %s.
[YYYY-MM-DDTHH:MM:SS] wcp:Patch ERROR wcp Not all patches were applied. Latest applied patch is 1
[YYYY-MM-DDTHH:MM:SS] wcp:Patch ERROR vmware_b2b.patching.executor.hook_executor Patch hook 'wcp:Patch' failed.
Traceback (most recent call last):
  File "/storage/seat/software-updatepwoebzhn/stage/scripts/patches/py/vmware_b2b/patching/executor/hook_executor.py", line 74, in executeHook
    executionResult = systemExtension(args)
  File "/storage/seat/software-updatepwoebzhn/stage/scripts/patches/libs/sdk/extensions.py", line 106, in __call__
    result = self.extension(*args)
  File "/storage/seat/software-updatepwoebzhn/stage/scripts/patches/libs/sdk/extensions.py", line 123, in _func
    return func(*args)
  File "/storage/seat/software-updatepwoebzhn/stage/scripts/patches/payload/components-script/wcp/__init__.py", line 225, in doPatching
    doIncrementalPatching(current_version)
  File "/storage/seat/software-updatepwoebzhn/stage/scripts/patches/payload/components-script/wcp/__init__.py", line 343, in doIncrementalPatching
    raise user_error
patch_errors.UserError: Failed to apply patch roles_groups_users! Error: {
    "detail": [
        {
            "id": "install.ciscommon.command.errinvoke",
            "translatable": "An error occurred while invoking external command : '%(0)s'",
            "args": [
                "Command: ['/usr/lib/vmware-vmca/bin/certool', '--server=VCENTER_FQDN', '--genCIScert', '--dataencipherment', '--privkey=/tmp/wcp_00000000.priv', '--cert=/tmp/wcp_000000.crt', '--Name=wcp']\nStderr: "
            ],
            "localized": "An error occurred while invoking external command : 'Command: ['/usr/lib/vmware-vmca/bin/certool', '--server=VCENTER_FQDN', '--genCIScert', '--dataencipherment', '--privkey=/tmp/wcp_0000000.priv', '--cert=/tmp/wcp_0000000.crt', '--Name=wcp']\nStderr: '"
        }
    ],
    "componentKey": null,
    "problemId": null,
    "resolution": null
}.
[YYYY-MM-DDTHH:MM:SS] ERROR vmware_b2b.patching.phases.patcher Patch hook Patch got ComponentWrapperError.
Traceback (most recent call last):
  File "/storage/seat/software-updatepwoebzhn/stage/scripts/patches/py/vmware_b2b/patching/phases/patcher.py", line 208, in patch
    _patchComponents(ctx, userData, statusAggregator.reportingQueue)
  File "/storage/seat/software-updatepwoebzhn/stage/scripts/patches/py/vmware_b2b/patching/phases/patcher.py", line 90, in _patchComponents
    executeComponentHook(Hook.Patch, ctx, c, userData, reportingQueue)
  File "/storage/seat/software-updatepwoebzhn/stage/scripts/patches/py/vmware_b2b/patching/executor/execution_facade.py", line 97, in executeComponentHook
    result = executeHook(c.patchScript, hook, args,
  File "/storage/seat/software-updatepwoebzhn/stage/scripts/patches/py/vmware_b2b/patching/executor/execution_facade.py", line 53, in executeHook
    result = executor.executeHook(scriptFile, hook, args, reportQueue, reportIdentifier)
  File "/storage/seat/software-updatepwoebzhn/stage/scripts/patches/py/vmware_b2b/patching/executor/hook_executor_process.py", line 119, in executeHook
    raise ex
patch_errors.ComponentError
[YYYY-MM-DDTHH:MM:SS] WARNING root stopping status aggregation...
[YYYY-MM-DDTHH:MM:SS] ERROR __main__ Patch vCSA failed

Environment

vCenter Server

Cause

This issue occurs because the vCenter Server's computer account is missing from the DCAdmins group within the VMDIR LDAP directory.
Without this group membership, the patching workflow lacks the required permissions to generate a new certificate for the wcp solution user using the certool utility.

Resolution

• To resolve this issue, you must manually add the vCenter Server's FQDN to the DCAdmins LDAP group.
• WARNING: Before proceeding, ensure you have a valid snapshot of the vCenter Appliance VM.

Important Pre-requisites:

Before running the commands below, you must identify and substitute two values specific to your environment:

<VCFQDN>: Your vCenter Server's Fully Qualified Domain Name (e.g., vcenter.corp.local).
<SSO_DOMAIN> components: Your SSO domain (default is vsphere.local). In the LDAP commands, this is formatted using domain components. For example, if your SSO domain is vsphere.local, you will use dc=vsphere,dc=local.

Step-by-Step Instructions:

• Connect to the vCenter Server Appliance via SSH and log in as the root user.
• Verify your exact SSO Domain name by running the following command:

/usr/lib/vmware-vmafd/bin/vmafd-cli get-domain-name --server-name localhost

• Run the following ldapmodify command to inject the vCenter computer account into the DCAdmins group.

Note: Carefully replace <VCFQDN>, <SSO_DOMAIN_PART1>, and <SSO_DOMAIN_PART2> with your environment's details. You will be prompted to enter the SSO Administrator password.

/opt/likewise/bin/ldapmodify -x -D cn=Administrator,cn=Users,dc=<SSO_DOMAIN_PART1>,dc=<SSO_DOMAIN_PART2> -W <<EOF
dn: CN=DCAdmins,cn=Builtin,dc=<SSO_DOMAIN_PART1>,dc=<SSO_DOMAIN_PART2>
changetype: modify
add: member
member: cn=<VCFQDN>,ou=Domain Controllers,dc=<SSO_DOMAIN_PART1>,dc=<SSO_DOMAIN_PART2>
EOF

• Verify the command output says modifying entry "CN=DCAdmins,cn=Builtin...".
• Retry the patching.