NSX Compute Manager 'Connection Status' Down

Products

VMware NSX VMware vCenter Server VMware NSX Data Center for vSphere

Issue/Introduction

Compute Manager 'Connection Status' in NSX shows Down.

If you attempt to edit the Compute Manager and click 'Save', you may encounter an error similar to one of the following:

Computer manager <name> with Id <ID> connection config is invalid. Edit Hostname and provide computer manager credentials. (Error code: 7055)
Certificate Chain of Computer Manager <name> is invalid. Please check Issuer and subject in the chain. (Error code: 90204)

The last inventory update date could match these scenarios:
- Environment may have been upgraded recently (NSX-T Data Center and/or vCenter).
- vCenter Certificates may have been changed recently.

You see messages similar to the following in the /var/log/cm-inventory/cm-inventory.log file on the NSX manager node:

<timestamps>  INFO http-nio-127.0.0.1-7443-exec-2 NsxTrustManagerBinding - SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="cm-inventory"] Try create  TrustManager of type PKIX
<timestamps>  WARN http-nio-127.0.0.1-7443-exec-2 VcUtilsImpl - SYSTEM [nsx@6876 comp="nsx-manager" level="WARNING" subcomp="cm-inventory"] IOException occurred
javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: java.security.cert.CertPathBuilderException: Unable to find certificate chain.

If the vCenter Server certificate was recently replaced with a CA certificate, you may see messages similar to the following in the /var/log/cm-inventory/cm-inventory.log file on the NSX manager node:

<timestamps> ERROR http-nio-127.0.0.1-7443-exec-2 VcPlugin 4732 SYSTEM [nsx@6876 comp="nsx-manager" errorCode="MP40219" level="ERROR" reqId="11111111-2222222-7e7e7e7e7" subcomp="cm-inventory" username="admin"] Certificate of Vc example.com is invalid. It might be caused by issuer not being same as subject of next certificate in certificate chain.

If you attempt to add a new computer manager to an existing NSX cluster, you may see messages similar to the following in the /var/log/cm-inventory/cm-inventory.log file on the NSX manager node:

<timestamps> ERROR http-nio-127.0.0.1-7443-exec-3 VcPlugin - SYSTEM [nsx@6876 comp="nsx-manager" errorCode="MP40106" level="ERROR" subcomp="cm-inventory"] Unable to login with username password for <IP/FQDN>
com.vmware.vim.vmomi.client.exception.SslException: javax.net.ssl.SSLHandshakeException: sun.security.validator.ValidatorException: PKIX path building failed: java.security.cert.CertPathBuilderException: No issuer certificate for certificate in certification path found.

Environment

VMware NSX-T Data Center
VMware NSX

VMware vCenter Server

Cause

The Connection Status is down due to an invalid or incorrect vCenter Server Machine SSL certificate chain.

A correct chain consists of:

Server/Leaf Certificate (vCenter Server)
Intermediate Certificate
Root Certificate

Resolution

The vCenter Machine SSL certificate chain needs to be checked and fixed.

The following steps can be used:

Obtain the Machine SSL certificate chain from vCenter Server using below command:
```
openssl s_client -showcerts -debug -connect <VC-IP>:443
```
Validate the certificate chain using any certificate checking resource. One recommended resource is KeyCDN Certificate Checker.
Chain the certificate correctly using the format Leaf/Server->Intermediate->Root
Use a file transfer utility to copy the correct certificate chain to the /tmp directory on the vCenter Server
Follow Replace VC Machine SSL certificate with Custom CA Signed Certificate
Edit the Compute Manager and click 'Save'.
Confirm the Connection Status shows 'Up'.

Note: When a CA certificate is in use on the vCenter Server, you must engage the CA vendor to issue a new certificate.

Additional Information

This issue can also occur if the vCenter Server certificate contains illegal or malformed characters. These issues may remain undetected under normal conditions.

However, certain validation processes in NSX can identify these malformed fields, resulting in the above errors when attempting to connect to the vCenter Server.

To identify if the vCenter Server certificate contains any problematic fields, follow these steps:

1. Export the vCenter Server certificate and save it to a file:

openssl s_client -connect localhost:443 -showcerts </dev/null 2>/dev/null | openssl x509 -outform PEM > vcenter.cer

2. View the certificate in text format and look for any unusual or unexpected values:

openssl x509 -in vcenter.cer -text -noout

3. Parse the certificate’s ASN.1 structure to check for additional problematic values:

openssl asn1parse -in vcenter.cer -i

Replacing vSphere Certificates

After VMware vCenter Server certificate is replaced, compute manager connection is "Down" on NSX UI