SDDC components are disconnected in Password Management, remediation fails with "javax.net.ssl.SSLHandshakeException: PKIX path validation failed"

Products

• VMware SDDC Manager

Issue/Introduction

• Several components within the SDDC such as ESXi, VxRail Manager, and vCenter are appearing as disconnected in the SDDC UI > Password Management.
• Password remediation is failing with the below error message:

Cause: javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors

Environment

VCF 5.2.1.1

Cause

• The issue is likely caused due to communication issues between the host and the SDDC Manager, which prevented the validation of ESXi host certificates, as indicated by the error.

/var/log/vmware/vcf/operationsmanager/operationsmanager.log

{"errorCode":"PASSWORD_MANAGER_VALIDATE_ESXI_CREDENTIALS_FAILED","arguments":["example.hostname.com"],"errorMessage":"javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors","referenceToken":"#####","remediationMessage":"Please verify that the account is active and is not locked, you might need to fix the workflow(s) for resources marked in error state. If the password of the account has expired, manually reset the password in the product and then perform a REMEDIATE operation in the SDDC Manager, to update its stored copy of the password."}

YYYY-MM-DD hh:mm:ss DEBUG [vcf_om,0000000000000000,0000] [c.v.v.s.t.DynamicTrustManager,reactor-http-nio-4] Trying to reload trusted certificates and recheck chain [email protected], CN=<ESXI FQDN>, OU=VMware Engineering, O=VMware, L=Palo Alto, ST=California, C=US
YYYY-MM-DD hh:mm:ss DEBUG [vcf_om,0000000000000000,0000] [c.v.v.s.t.DynamicTrustManager,reactor-http-nio-4] Custom Trust Strategy initialized.
YYYY-MM-DD hh:mm:ss WARN  [vcf_om,0000000000000000,0000] [r.n.http.client.HttpClientConnect,reactor-http-nio-4] [b4e5509d, L:/<IP Address>:60368 - R:<ESXI FQDN>/<IP Address>:443] The connection observed an error
javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: Path does not chain with any of the trust anchors

Resolution

NOTE: Take backup/snapshot of the SDDC manager.

• Renew the ESXi host certificates in the cluster using the vSphere Client or vCenter. Renew or Refresh ESXi Certificates
  • (Optional) After renewal, disconnect each host from vCenter and reconnect it to update trust and apply new certificates.
• Verify that all hosts show as “connected” in vCenter after the certificate renewal.
• Once all hosts are connected successfully, initiate password remediation for affected components in SDDC Manager.