NSX UI/API not accesible after CA certificate update (error seen: SSL_ERROR_SYSCALL)

Products

• VMware NSX

Issue/Introduction

• One or more NSX UI not accessible with IP or FQDN. But all services and cluster state show Up/Stable.
• Checking from one of the working NSX, it shows the new certificate has been applied successfully.
• TLS handshakes to the node on TCP/443 fails even locally.
• In curl you may see the following error when attempting to make an API call against the node:
  • curl: (35) OpenSSL SSL_connect: SSL_ERROR_SYSCALL in connection to <FQDN/IP>:443
• Running curl -kv https://<FQDN/IP> on the impacted Manager shows the certificate being pointed to old one or no certificate at all.
• NSX UI log support bundles may fail to any and all manager nodes except the one that currently holds the VIP.
• Running /etc/init.d/envoy status as root on the NSX manager node reveals log lines similar to the following reported by systemd:
  /home/secureall/secureall/.store/.tomcat_cert.pem should start with -----BEGIN 
• In /var/log/proxy/envoy.log you see log lines similar to as follows:
         https-node-v4-local: Failed to load certificate chain from <inline>

Environment

NSX 4.x

Resolution

This is currently impacting 4.0.x release and will be fixed in a future release.

Workaround:
Restart Envoy, confirm it is now running, and validate that the node can be reached now via IP/FQDN when browsing to it.
/etc/init.d/envoy restart
/etc/init.d/envoy status

Running curl -kv https://<FQDN/IP> against the impacted will now show the updated certificate and the UI should come UP.