Compute Manager 'Connection Status' goes Down after Upgrade from 3.X or 4.0.1 to 4.1.

Products

• VMware NSX

Issue/Introduction

• From NSX UI under System>Fabric>Compute managers you see "Connection Status" as Down.
• If you take a closer look at "Last Inventory Update" matches the date when NSX was Upgrade.

• From logs /var/log/cm-inventory/cm-inventory.log you see the following : 

2023-05-05T20:57:33.609Z  WARN InventoryFetcher-5c9f6ff1-71cf-####-####-########b72 CrlWebFetcher 4518 SYSTEM [nsx@6876 comp="nsx-manager" level="WARNING" subcomp="cm-inventory"] Couldn't get LDAP context from URI ldap:///CN=srv-root-CA(1),CN=######,CN=CDP,CN=Public%20Key%20Services,CN=Services,CN=Configuration,DC=example,DC=com?certificateRevocationList?base?objectClass=cRLDistributionPoint
javax.naming.CommunicationException: ldapserver.example.com.:389
        at com.sun.jndi.ldap.Connection.<init>(Connection.java:243) ~[?:1.8.0_352]
        at com.sun.jndi.ldap.LdapClient.<init>(LdapClient.java:137) ~[?:1.8.0_352]
        at com.sun.jndi.ldap.LdapClient.getInstance(LdapClient.java:1615) ~[?:1.8.0_352]
        at com.sun.jndi.ldap.LdapCtx.connect(LdapCtx.java:2849) ~[?:1.8.0_352]
        at com.sun.jndi.ldap.LdapCtx.<init>(LdapCtx.java:347) ~[?:1.8.0_352]
TRUNCATED….
Caused by: java.net.ConnectException: Connection timed out (Connection timed out)
        at java.net.PlainSocketImpl.socketConnect(Native Method) ~[?:1.8.0_352]
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350) ~[?:1.8.0_352]
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206) ~[?:1.8.0_352]
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188) ~[?:1.8.0_352]
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) ~[?:1.8.0_352]
        at java.net.Socket.connect(Socket.java:607) ~[?:1.8.0_352]
        at java.net.Socket.connect(Socket.java:556) ~[?:1.8.0_352]
        at java.net.Socket.<init>(Socket.java:452) ~[?:1.8.0_352]
        at java.net.Socket.<init>(Socket.java:229) ~[?:1.8.0_352]
        at com.sun.jndi.ldap.Connection.createSocket(Connection.java:380) ~[?:1.8.0_352]
        at com.sun.jndi.ldap.Connection.<init>(Connection.java:220) ~[?:1.8.0_352]
        ... 47 more
		

• Using the following API call to the NSX-T manager, we see CRL checking is enabled:

GET https://<manager>/api/v1/global-configs/SecurityGlobalConfig
Result:
...
" crl_checking_enabled": true,
...

Note: The preceding log excerpts are only examples. Date, time, and other variables may vary depending on your environment.

 

Environment

VMware NSX-T Data Center
VMware NSX 4.0.0.1

Cause

Starting on 4.1 version we needed to set crl checking to 'true' for Compliance.

NSX does not support LDAP hosting of CDPs. This because most LDAP CDPs don't specify the host, like in "ldap:///... (There's supposed to be a host-name between the second and third '/')

ldap://<GAP,HERE>/CN=srv-root-CA(1),CN=vcsa-01,CN=CDP,CN=Public%20Key%20Services,CN=Services,CN=Configuration,DC=example,DC=com?certificateRevocationList?base?objectClass=cRLDistributionPoint

 

 

Resolution

No resolution available, need to move away from LDAP protocol when CRL distribution point (CDP) is set on vCenter Certificate.

Workaround:

• A- Disable 'crl_checking_enabled' via API : 

GET https://{{ip}}/policy/api/v1/infra/security-global-config

JSON response: 
{
    "crl_checking_enabled": true,
    "ca_signed_only": false,
    "eku_checking_enabled": true,
    "id": "e19b6c5b-9065-####-####-########35f",
    "_create_time": 1679339007871,
    "_create_user": "system",
    "_last_modified_time": 1679339007871,
    "_last_modified_user": "system",
    "_protection": "NOT_PROTECTED",
    "_revision": 0
}

Then use that as input for:

PUT https://{{ip}}/policy/api/v1/infra/security-global-config

"crl_checking_enabled": false

 

• B- Get a new vCenter Certificate without CDP attribute enable(in which case no CRL checking will be performed)
• C- Get a new vCenter  certificate that uses HTTP(s) address with the CDP. 

crlDistributionPoints=URI:http://example.com/crl.pem

This is the most common solution. A way to verify the CDP is to log into an NSX instance and do "wget <cdp-link>" and see if it manages to fetch it.