Unable to generate CSR from SDDC for NSX-T Manager with expired certificates.

Products

• VMware Cloud Foundation

Issue/Introduction

Symptoms:

CSR Generation fails for NSX-T Manager with expired certificates.

/var/log/vmware/vcf/operationsmanager/operationsmanager.log
==========================================================

2022-09-26T17:57:37.207+0000 ERROR [vcf_om,8c1b7bb780e54692,831b] [c.v.v.c.n.NsxTManagerCertificatePluginService,om-exec-16] CSR generation failed for vip-nsx-mgmt.vrack.vsphere.local: I/O error on GET request for "https://vip-nsx-mgmt.vrack.vsphere.local/api/v1/cluster/nodes/deployments": PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed; nested exception is javax.net.ssl.SSLHandshakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed 2022-09-26T17:57:37.208+0000 ERROR [vcf_om,8c1b7bb780e54692,140f] [c.v.v.c.n.NsxTManagerCertificatePlugin,om-exec-17] Unable to generate CSR for resource: nsx-mgmt-1.vrack.vsphere.local com.vmware.vcf.certmgmt.exceptions.CsrGenerationException: I/O error on GET request for "https://nsx-mgmt-1.vrack.vsphere.local/api/v1/cluster/nodes/ deployments": PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed; nested exception is javax.net.ssl.SSLHand shakeException: PKIX path validation failed: java.security.cert.CertPathValidatorException: validity check failed at com.vmware.vcf.certmgmt.nsx_t.NsxTManagerCertificatePluginService.buildCsrApiRequest(NsxTManagerCertificatePluginService.java:411) at 

Environment

VMware Cloud Foundation 4.x

Cause

SDDC is unable to communicate with NSX-T to generate the CSRs if the NSX-T Manager certificates are expired.

Resolution

Workaround:

Apply the original vCenter CA signed certificates(If still valid) to the NSX-T Managers/VIP from the NSX-T API.

1. Log into the NSX-T UI

System > Certificates

2. Collect the certificate IDs and FQDNs for NSX-T Managers and the VIP.
(You need to collect the ID - click on it to copy)

i.e

vip-nsx-mgmt.vrack.vsphere.local 519ce43c-ae3b-4e62-9c2e-f5745b36280b
nsx-mgmt-1.vrack.vsphere.local 513ab6e9-484a-4c23-a806-8d5577f9108c

3.Once you have collected all of the information you can apply the certificates to ALL of the NSX-T Managers and the VIP. SSH with root into one of the NSX-T Managers or SDDC Manager and run the below commands.

NSX-T VIP

curl -k -u admin -X POST 'https://vip-nsx-mgmt.vrack.vsphere.local/api/v1/cluster/api-certificate?
action=set_cluster_certificate&certificate_id=519ce43c-ae3b-4e62-9c2e-f5745b36280b'

NSX-T Managers (Change the FQDN and certificate ID for each manager)

curl -k -u admin -X POST 'https://nsx-mgmt01.vrack.vsphere.local/api/v1/node/services/http?action=apply_certificate&certificate_id=513ab6e9-484a-4c23-a806-8d5577f9108c'

4.Now refresh the SDDC UI and the certificates should show valid and the CSR creation will succeed.

Additional Information

If the vCenter CA signed certificates are expired. Run the script in KB 89921 to generate new vCenter signed certificates for the NSX-T Managers and VIP.

https://kb.vmware.com/s/article/89921