Error: "ESXi VASA client certificate provision has failed" alert reported post ESXi upgrade from version 7 to 8
search cancel

Error: "ESXi VASA client certificate provision has failed" alert reported post ESXi upgrade from version 7 to 8

book

Article ID: 386738

calendar_today

Updated On:

Products

VMware vSphere ESX 7.x VMware vSphere ESX 8.x VMware vCenter Server 8.0

Issue/Introduction

  • After upgrading ESXi from version 7 to 8, the following alert was seen on the ESXi host through vCenter:

  • The issue prevents the ESXi host from being added to vCenter when the vpxd.certmgmt.mode is set to VMCA.
  • If the vpxd.certmgmt.mode is switched to thumbprint, the ESXi host is successfully added to vCenter, indicating a certificate management issue when using the VMCA option.

Validation:

  • vCenter - /var/log/vmware/vmware-sps/sps.log: The "certificate_unknown(46)" error code suggests that the SSL certificate presented by the ESXi host is not trusted by the system, or the certificate chain is incomplete
    YYYY-MM-DDT05:28:57.171-06:00 [pool-29-thread-1] WARN  opId=########-###-####-##-##:########-## com.vmware.vim.sms.provider.vasa.VersionHandler - [isLegacyProvider] Failed to retrive version information from provider - https://example.com:9080/version.#ml: certificate_unknown(46)
    YYYY-MM-DDT05:28:57.171-06:00 [pool-29-thread-1] WARN  opId=########-###-####-##-##:########-## com.vmware.vim.sms.provider.vasa.VasaProviderUtils - [isValidatorexception]: Unknown certificate
    YYYY-MM-DDT05:46:01.061-06:00 [pool-186-thread-26] WARN  opId=########-###-####-##-##:########-## com.vmware.vim.sms.provider.ProviderFactory - provisionEsxClientCertificates failed for host-1##, com.vmware.certificate.VMCAE#ception: UNKNOWN
    YYYY-MM-DDT05:46:01.067-06:00 [pool-186-thread-22] INFO  opId=########-###-####-##-##:########-## com.vmware.vim.sms.provider.vasa.cert.CertificateAuthority - Timer stopped: getCAsignedCertificateInt, Time taken: 324 ms.
    YYYY-MM-DDT05:46:01.067-06:00 [pool-186-thread-22] ERROR opId=########-###-####-##-##:########-## com.vmware.vim.sms.provider.vasa.cert.CertificateAuthority - Failed to get a VMCA signed certificate for CSR. Error : 5, Message : UNKNOWN
    YYYY-MM-DDT05:46:01.067-06:00 [pool-186-thread-22] INFO  opId=########-###-####-##-##:########-## com.vmware.vim.sms.provider.ProviderFactory - Timer stopped: provisionAndAddCASignedEsxClientCertificate, Time taken: 756 ms.
    YYYY-MM-DDT05:46:01.068-06:00 [pool-186-thread-22] ERROR opId=########-###-####-##-##:########-## com.vmware.vim.sms.provider.ProviderFactory - provisionCASignedEsxClientCertificate failed
    com.vmware.vim.sms.fault.CertificateException: Failed to get a VMCA signed certificate for CSR. Error : 5, Message : UNKNOWN
  • vCenter - /var/log/vmware/vpxd/vpxd.log: The error code "ERROR_ACCESS_DENIED (5)" indicates that the user does not have the necessary permissions to access the resource or perform the operation:
    YYYY-MM-DDT00:24:57.583-06:00 error vpxd[713949] [Originator@6876 sub=certmgrLogger opID=########-###-####-##-##:########-##] Unable to get signed certificate forhost name 'example.com' ip '###.##.##.##': Error: Operation failed with error = ERROR_ACCESS_DENIED (5) (5)
    YYYY-MM-DDT00:24:57.595-06:00 error vp#d[713949] [Originator@6876 sub=Default opID=########-###-####-##-##:########-##] [Vp#LRO] -- ERROR task-55025 -- 522aba30-####-###-####-359133f5d4a1(5201e02a-####-####-####-53a8ddb5b150) -- host-1## -- vim.HostSystem.reconnect: :vmodl.fault.SystemError
    --> Result:
    --> (vmodl.fault.SystemError) {
    -->   faultCause = (vmodl.MethodFault) null,
    -->   faultMessage = ,
    -->   reason = "Unable to get signed certificate forhost name 'example.com' ip '###.##.##.##': Error: Operation failed with error = ERROR_ACCESS_DENIED (5) (5)"
    -->   msg = ""
    
    YYYY-MM-DDT02:14:22.297-06:00 warning vpxd[803986] [Originator@6876 sub=HttpConnectionPool-000001 opID=########-###-####-##-##:########-##] Failed to get pooled connection; <cs p:00007f808c8f74d0, TCP:example.com:443>, SSL(<io_obj p:0x00007f805c08e610, h:100, <TCP '###.##.##.## : 51260'>, <TCP '###.##.##.## : 443'>>), duration: 5msec, N7Vmacore3Ssl18SSLVerifyE#ceptionE(SSL E#ception: Verification parameters:
    --> PeerThumbprint: A0:##:03:##:2B:##:2A:3A:##:24:A1:FC:##:82:E4:##:44:##:4D:CF
    --> ExpectedThumbprint:
    --> ExpectedPeerName: example.com
    --> The remote host certificate has these problems:
    -->
    --> * unable to get local issuer certificate)
    
  • vCenter - var/log/vmware/vmcad/vmcad log: Operation with error in "vmcad" hinting to an authorization problem.
    YYYY-MM-DDT04:22:56.342180-06:00 info vmcad YYYY-MM-DDT10:22:56.342 [vmcad][INFO] Checking upn: cn=CAAdmins,cn=Builtin,dc=vsphere,dc=local against CA admin group: vcenter@vsphere.local
    YYYY-MM-DDT04:22:56.342187-06:00 info vmcad YYYY-MM-DDT10:22:56.342 [vmcad][INFO] Checking user's group: cn=Administrators,cn=Builtin,dc=vsphere,dc=local against CA admin group: cn=CAAdmins,cn=Builtin,dc=vsphere,dc=local
    YYYY-MM-DDT04:22:56.342190-06:00 info vmcad YYYY-MM-DDT10:22:56.342 [vmcad][INFO] Checking user's group: CN=SystemConfiguration.Administrators,DC=vsphere,DC=local against CA admin group: cn=CAAdmins,cn=Builtin,dc=vsphere,dc=local
    YYYY-MM-DDT04:22:56.342194-06:00 info vmcad YYYY-MM-DDT10:22:56.342 [vmcad][INFO] Checking user's group: CN=LicenseService.Administrators,DC=vsphere,DC=local against CA admin group: cn=CAAdmins,cn=Builtin,dc=vsphere,dc=local
    YYYY-MM-DDT04:22:56.342196-06:00 info vmcad YYYY-MM-DDT10:22:56.342 [vmcad][WARNING] [lotus/vmca/service/auth.c:VMCALdapAccessCheck:92] error code: 0x00000005
    YYYY-MM-DDT04:22:56.342198-06:00 info vmcad YYYY-MM-DDT10:22:56.342 [vmcad][INFO] VMCACheckAccessKrb: Access denied as user is not administrator
    YYYY-MM-DDT04:22:56.342200-06:00 info vmcad YYYY-MM-DDT10:22:56.342 [vmcad][WARNING] [lotus/vmca/service/rpcserv.c:VMCACheckAccess:103] error code: 0x00000005
    YYYY-MM-DDT04:22:56.342202-06:00 info vmcad YYYY-MM-DDT10:22:56.342 [vmcad][WARNING] [lotus/vmca/service/rpcserv.c:RpcVMCAGetSignedCertificate:364] error code: 0x00000005
    YYYY-MM-DDT04:22:56.342204-06:00 info vmcad YYYY-MM-DDT10:22:56.342 [vmcad][INFO] [RPC] Exiting RpcVMCAGetSignedCertificate, Status = 5

Environment

  • VMware vCenter Server 7.x
  • VMware vCenter Server 8.x
  • VMware vSphere ESXi 7.x
  • VMware vSphere ESXi 8.x

Cause

  • The issue is caused by the absence of the DCAdmins group in the CAAdmins SSO group.
  • This missing group leads to authorization issues within vCenter Server operations, which prevents the ESXi host from obtaining the necessary VMCA-signed certificate.
  • The following snippet shows that the DCAdmins group should be part of the "Users & Groups" section:
  • The following snippet shows that the DCAdmins group should also be included in the CAAdmins SSO group:

Resolution

If the issue aligns with the logs, contact Broadcom Support for further investigation. 

Additional Information