After Upgrading ESXi from Version 7 to 8, the Following Alert is Reported: "Error: ESXi VASA Client Certificate Provision Has Failed."
search cancel

After Upgrading ESXi from Version 7 to 8, the Following Alert is Reported: "Error: ESXi VASA Client Certificate Provision Has Failed."

book

Article ID: 386738

calendar_today

Updated On:

Products

VMware vSphere ESXi VMware vCenter Server

Issue/Introduction

Symptoms:

  • After upgrading ESXi from version 7 to 8, the following alert was seen on the ESXi host through vCenter:

 

  • The issue prevents the ESXi host from being added to vCenter when the vpxd.certmgmt.mode is set to VMCA.
  • If the vpxd.certmgmt.mode is switched to thumbprint, the ESXi host is successfully added to vCenter, indicating a certificate management issue when using the VMCA option.
Validation:
  • /var/log/vmware/vmware-sps/sps.log: The "certificate_unknown(46)" error code suggests that the SSL certificate presented by the ESXi host is not trusted by the system or the certificate chain is incomplete

YYYY-MM-DDT05:28:57.171-06:00 [pool-29-thread-1] WARN  opId=m67sxxbm-1707-auto-1bg-h5:70000548 com.vmware.vim.sms.provider.vasa.VersionHandler - [isLegacyProvider] Failed to retrive version information from provider - https://XXX.YYY.com:9080/version.xml: certificate_unknown(46)
YYYY-MM-DDT05:28:57.171-06:00 [pool-29-thread-1] WARN  opId=m67sxxbm-1707-auto-1bg-h5:70000548 com.vmware.vim.sms.provider.vasa.VasaProviderUtils - [isValidatorexception]: Unknown certificate
YYYY-MM-DDT05:46:01.061-06:00 [pool-186-thread-26] WARN  opId=m67sxxbm-2603-auto-20c-h5:70000772 com.vmware.vim.sms.provider.ProviderFactory - provisionEsxClientCertificates failed for host-1xx, com.vmware.certificate.VMCAException: UNKNOWN
YYYY-MM-DDT05:46:01.067-06:00 [pool-186-thread-22] INFO  opId=m67sxxbm-2603-auto-20c-h5:70000772 com.vmware.vim.sms.provider.vasa.cert.CertificateAuthority - Timer stopped: getCAsignedCertificateInt, Time taken: 324 ms.
YYYY-MM-DDT05:46:01.067-06:00 [pool-186-thread-22] ERROR opId=m67sxxbm-2603-auto-20c-h5:70000772 com.vmware.vim.sms.provider.vasa.cert.CertificateAuthority - Failed to get a VMCA signed certificate for CSR. Error : 5, Message : UNKNOWN
YYYY-MM-DDT05:46:01.067-06:00 [pool-186-thread-22] INFO  opId=m67sxxbm-2603-auto-20c-h5:70000772 com.vmware.vim.sms.provider.ProviderFactory - Timer stopped: provisionAndAddCASignedEsxClientCertificate, Time taken: 756 ms.
YYYY-MM-DDT05:46:01.068-06:00 [pool-186-thread-22] ERROR opId=m67sxxbm-2603-auto-20c-h5:70000772 com.vmware.vim.sms.provider.ProviderFactory - provisionCASignedEsxClientCertificate failed
com.vmware.vim.sms.fault.CertificateException: Failed to get a VMCA signed certificate for CSR. Error : 5, Message : UNKNOWN

 

  • /var/log/vmware/vpxd/vpxd.log: The error code "ERROR_ACCESS_DENIED (5)" indicates that the user does not have the necessary permissions to access the resource or perform the operation:

YYYY-MM-DDT00:24:57.583-06:00 error vpxd[713949] [Originator@6876 sub=certmgrLogger opID=m6envwau-644-auto-hx-h5:70000201-bb] Unable to get signed certificate forhost name 'XXXX.YYY.com' ip 'xxx.xx.xx.xx': Error: Operation failed with error = ERROR_ACCESS_DENIED (5) (5)
YYYY-MM-DDT00:24:57.595-06:00 error vpxd[713949] [Originator@6876 sub=Default opID=m6envwau-644-auto-hx-h5:70000201-bb] [VpxLRO] -- ERROR task-55025 -- 522aba30-822d-fc9d-eb3b-359133f5d4a1(5201e02a-c4c1-617e-b100-53a8ddb5b150) -- host-1xx -- vim.HostSystem.reconnect: :vmodl.fault.SystemError
--> Result:
--> (vmodl.fault.SystemError) {
-->    faultCause = (vmodl.MethodFault) null,
-->    faultMessage = <unset>,
-->    reason = "Unable to get signed certificate forhost name 'XXX.YYY.com' ip 'xxx.xx.xx.xx': Error: Operation failed with error = ERROR_ACCESS_DENIED (5) (5)"
-->    msg = ""

YYYY-MM-DDT02:14:22.297-06:00 warning vpxd[803986] [Originator@6876 sub=HttpConnectionPool-000001 opID=m6erpo18-548-auto-f9-h5:70000142-7c] Failed to get pooled connection; <cs p:00007f808c8f74d0, TCP:XXX.YYY.com:443>, SSL(<io_obj p:0x00007f805c08e610, h:100, <TCP 'xxx.xx.xx.xx : 51260'>, <TCP 'xxx.xx.xx.xx : 443'>>), duration: 5msec, N7Vmacore3Ssl18SSLVerifyExceptionE(SSL Exception: Verification parameters:
--> PeerThumbprint: A0:xx:03:xx:2B:xx:2A:3A:xx:24:A1:FC:xx:82:E4:xx:44:xx:4D:CF
--> ExpectedThumbprint:
--> ExpectedPeerName: XXX.YYY.com
--> The remote host certificate has these problems:
-->
--> * unable to get local issuer certificate)

  • var/log/vmware/vmcad/vmcad logs: Operation with error in "vmcad" hinting to an authorization problem.

YYYY-MM-DDT04:22:56.342180-06:00 info vmcad YYYY-MM-DDT10:22:56.342 [vmcad][INFO] Checking upn: cn=CAAdmins,cn=Builtin,dc=vsphere,dc=local against CA admin group: [email protected]
YYYY-MM-DDT04:22:56.342187-06:00 info vmcad YYYY-MM-DDT10:22:56.342 [vmcad][INFO] Checking user's group: cn=Administrators,cn=Builtin,dc=vsphere,dc=local against CA admin group: cn=CAAdmins,cn=Builtin,dc=vsphere,dc=local
YYYY-MM-DDT04:22:56.342190-06:00 info vmcad YYYY-MM-DDT10:22:56.342 [vmcad][INFO] Checking user's group: CN=SystemConfiguration.Administrators,DC=vsphere,DC=local against CA admin group: cn=CAAdmins,cn=Builtin,dc=vsphere,dc=local
YYYY-MM-DDT04:22:56.342194-06:00 info vmcad YYYY-MM-DDT10:22:56.342 [vmcad][INFO] Checking user's group: CN=LicenseService.Administrators,DC=vsphere,DC=local against CA admin group: cn=CAAdmins,cn=Builtin,dc=vsphere,dc=local
YYYY-MM-DDT04:22:56.342196-06:00 info vmcad YYYY-MM-DDT10:22:56.342 [vmcad][WARNING] [lotus/vmca/service/auth.c:VMCALdapAccessCheck:92] error code: 0x00000005
YYYY-MM-DDT04:22:56.342198-06:00 info vmcad YYYY-MM-DDT10:22:56.342 [vmcad][INFO] VMCACheckAccessKrb: Access denied as user is not administrator
YYYY-MM-DDT04:22:56.342200-06:00 info vmcad YYYY-MM-DDT10:22:56.342 [vmcad][WARNING] [lotus/vmca/service/rpcserv.c:VMCACheckAccess:103] error code: 0x00000005
YYYY-MM-DDT04:22:56.342202-06:00 info vmcad YYYY-MM-DDT10:22:56.342 [vmcad][WARNING] [lotus/vmca/service/rpcserv.c:RpcVMCAGetSignedCertificate:364] error code: 0x00000005
YYYY-MM-DDT04:22:56.342204-06:00 info vmcad YYYY-MM-DDT10:22:56.342 [vmcad][INFO] [RPC] Exiting RpcVMCAGetSignedCertificate, Status = 5

Environment

VMware vCenter Server 7.x
VMware vCenter Server 8.x
VMware vSphere ESXi 7.x
VMware vSphere ESXi 8.x

Cause

  • The issue is caused by the absence of the DCAdmins group in the CAAdmins SSO group.
  • This missing group leads to authorization issues within vCenter Server operations, which prevents the ESXi host from obtaining the necessary VMCA-signed certificate.
  • The following snippet shows that the DCAdmins group should be part of the "Users & Groups" section:

  • The following snippet shows that the DCAdmins group should also be included in the CAAdmins SSO group:

Resolution

If the issue aligns with the logs, please contact Broadcom Support for further investigation. 

Additional Information

Powered by Wolken Software