EAM API call fails with CertificateNotTrustedFault or EAM agent has CertificateNotTrusted issue.
Article ID: 313402
Updated On:
Products
Issue/Introduction
Symptoms:
One or more of the following symptoms can be observed:
Symptom 1: EAM API call fails with CertificateNotTrustedFault.
In the case of CertificateNotTrustedFault, a log snippet similar to this below is added to the /var/log/vmware/eam/eam-api.log file.
2023-06-23T08:53:20.681Z | ERROR | vlsi | LocalizationFilter.java | 94 | API ERROR: EsxAgentManager.createAgency[opId=1196470828, sessionId=6823FB51]. Fault:
eam.fault.CertificateNotTrustedFault {
url = 'https://10.185.xx.xx:34591/vm/vm.ovf',
}
(eam.fault.CertificateNotTrustedFault) {
faultCause = com.vmware.vim.binding.vmodl.MethodFault: "com.vmware.eam.security.trust.NotTrusted: Suitable trust, not found!" caused by "org.bouncycastle.tls.TlsFatalAlert: certificate_unknown(46)" caused by "java.security.cert.CertificateException: Unable to construct a valid chain" caused by "java.security.cert.CertPathBuilderException: Unable to find certificate chain." Please follow KB 93130,
faultMessage = null,
url = https://10.185.xx.xx:34591/vm/vm.ovf
}
Symptom 2: EAM Agent has a CertificateNotTrusted issue.
The user can detect the issue using the vSphere UI or by checking the content of the /var/log/vmware/eam/eam.log file.
- The issue is presented in the ЕАМ UI (vCenter Client → Administration → vCenter Server Extensions → ESX Agent Manager → Monitor) in one of the following ways:
- The issue is presented in the /var/log/vmware/eam/eam.log file with a snippet similar to one of the following:
Snippet 1:
2023-06-23T09:38:37.872Z | DEBUG | host-82-0 | IssueHandlerBase.java | 539 | About to add unique issue: HostAgentIssueHandler:HostAgent(ID: 802a61b5-####-####-####-########f0d)
eam.issue.CertificateNotTrusted {
time = 2023-06-23 09:38:37,871,
description = <unset>,
key = 1,
agencyName = 'test-solution-2023-06-23T09:38:26.821+0000',
solutionId = 'eamUnitTest',
solutionName = 'eamUnitTest',
agency = 'Agency:e3d398f8-####-####-####-########423:null',
agent = 'Agent:802a61b5-####-####-####-########f0d:null',
agentName = '802a61b5-####-####-####-########f0d',
host = 'HostSystem:host-82:3b47f7fd-####-####-####-########33a',
hostName = '10.186.81.225',
url = 'https://10.93.xxx.xxx:8080/vm/vm.ovf',
}
Snippet 2:
2023-06-23T08:31:21.547Z | DEBUG | cluster-agent-1 | IssueHandlerBase.java | 539 | About to add unique issue: ClusterAgentIssueHandler:ClusterAgent(ID: 9325a7d2-####-####-####-########196)
eam.issue.cluster.agent.CertificateNotTrusted {
description = <unset>,
time = 2023-06-23 08:31:21,547,
key = 1,
agencyName = 'test-solution-2023-06-23T08:31:08.562+0000',
solutionId = 'eamUnitTest',
solutionName = 'eamUnitTest',
agency = 'Agency:843b60d1-####-####-####-########c6d:null',
agent = 'Agent:9325a7d2-####-####-####-########196:null',
cluster = 'ClusterComputeResource:domain-c56:3b47f7fd-####-####-####-########33a',
url = 'https://10.93.xxx.xxx:8080/vm/vm.ovf',
}
Environment
VMware vCenter Server 8.0
Cause
Any of the below mentioned cases can cause the issue:
-
The file server that hosts the OVF and/or VIB URLs of an EAM Agency uses an SSL certificate and:
-
There is an SSL certificate hostname mismatch, or
-
The SSL certificate is invalid or
-
The SSL certificate is self-signed, or
-
The SSL certificate is not trusted by the system. That is, the certificate is not signed by any of the root CA certificates of Photon S or VECS TRUSTED_ROOTS
-
The SSL certificate is provided via the EAM API (Agent.ConfigInfo.ovfSslTrust) and/or (Agent.ConfigInfo.vibSslTrust) or via the script /usr/lib/vmware-eam/bin/eam-utility.py and:
-
There is an SSL certificate hostname mismatch, or
-
The SSL certificate is invalid, or
-
The SSL certificate does not match the SSL certificate of the file server that hosts the OVF and/or VIB URLs of an EAM Agency.
Resolution
To resolve the issue please follow any one of the options mentioned below:
Option 1: Configure SSL trust via EAM API
Use EAM API properties Agent.ConfigInfo.ovfSslTrust and/or Agent.ConfigInfo.vibSslTrust to:
- Configure a leaf SSL certificate that is to be trusted for a specific VIB or OVF URL.
- Disable the SSL certificate verification when trying to access a specific OVF or VIB URL.
Option 2: Change the file server SSL certificate
Replace the file server SSL certificate with a valid one or such that is signed by any of Photon OS CAs or VECS TRUSTED_ROOTS CAs.
Option 3: Add a trusted root CA certificate to VECS
Add the root CA certificate signing the file server certificate to VECS TRUSTED_ROOTS.
Please refer Add a Trusted Root Certificate to the Certificate Store using the vSphere Client and vecs-cli Command Reference for more information.
Workaround:
Please follow one of the below mentioned options to configure the SSL trust via an EAM script.
Option 1: Configure a leaf SSL certificate that is to be trusted for a specific VIB or OVF URL
- Login to VCSA through SSH using root.
- Run the below command:
#/usr/lib/vmware-eam/bin/eam-utility.py install-cert <VIB/OVF URL>
Note:
- The operation above can be reverted by running: eam-utility.py uninstall-cert <VIB/OVF URL>
- The agency owner can also do the SSL trust configuration via the EAM API. In this case, it takes precedence over the configuration made via the script /usr/lib/vmware-eam/bin/eam-utility.py.
Option 2: Disable the SSL certificate verification for a specific VIB or OVF URL
- Login to VCSA through SSH using root.
- Run the below command:
#/usr/lib/vmware-eam/bin/eam-utility.py disable-trust <VIB/OVF URL>
Note:
- The operation above can be reverted by running: eam-utility.py enable-trust <VIB/OVF URL>
- The agency owner can also do the SSL trust configuration via the EAM API. In this case, it takes precedence over the configuration made via the script /usr/lib/vmware-eam/bin/eam-utility.py.
Feedback
