Alarm For Transport Node Certificate Expiration Approaching

Products

• VMware NSX

Issue/Introduction

Title: Alarm for transport_node_certificate_expiration_approaching
Event ID: transport_node_certificate_expiration_approaching
Alarm Description

• Purpose: Notify User that Transport Node Certificate expiration is approaching in 30 days or less.

• Impact: Transport Node can disconnect from Managers and not connect back again after the certificate expiry.

• Cause: Transport Node certificate expiration is approaching in 30 days or less.

Warning: This alarm must be addressed as soon as possible. Once the TN certificate expires, there is a grace period of 24 hours after which all impacted Edges and Hosts will be disconnected from NSX.

Environment

VMware NSX 4.1.x and 4.2.0

Cause

On versions NSX 4.1.x and 4.2.0, Edge and Host Transport Nodes are instantiated using a certificate with validity period of 825 days.
NSX-T 3.x and NSX 4.2.1 and higher create Transport Nodes using a certificate with validity period of 10 years.
The Transport Node certificate used at create time is not replaced on upgrade. 
Any Edge that may have been deployed on these versions or any Hosts prepared or re-prepared on these versions will have this shorter validity period certificate.

Resolution

For NSX versions from 4.1.0 through to 4.2.0 inclusive:

Check the connection status of the Transport Node on the NSX UI, System -> Fabric -> Hosts(ESXi)/Nodes(Edge)

Transport Node has an expired or expiring certificate but is still connected to NSX:

The CARR script can be used to replace the TN certificates. See the section "Transport Node Certificates" on Using Certificate Analyzer, Results and Recovery (CARR) Script to fix certificate related issues in NSX.

Transport Node certificate has expired and TN is in a disconnected state in NSX:

1. 1. Open an SSH session with the Transport Node as root
   2. Empty Transport Node certificate and private key
      
      cat /dev/null > /etc/vmware/nsx/host-cert.pem
cat /dev/null > /etc/vmware/nsx/host-privkey.pem
      
      

   3. Generate a new self-signed TN certificate and key, following one of the procedures below
      
      
      For NSX 4.1.x versions prior to 4.1.2.5:
      a)  Create a temporary openssl config file from the existing openssl config

      cat /etc/vmware/nsx/openssl-proxy.cnf > /tmp/tmp-openssl-proxy.cnf

      b) Extract the host UUID and add it to the temporary openssl config

      echo "UID = $(grep -o '<uuid>[^<]*' /etc/vmware/nsx/host-cfg.xml | sed 's/<uuid>//')" >> /tmp/tmp-openssl-proxy.cnf
      
      

      c) Add extension in the temporary openssl config

      
      echo -e "[ req_ext ]\nbasicConstraints     = CA:FALSE\nextendedKeyUsage     = clientAuth\nsubjectKeyIdentifier = hash\nauthorityKeyIdentifier = keyid,issuer" >> /tmp/tmp-openssl-proxy.cnf

      d)  Replace the certificate, where the -days parameter specifies 3650 days (10 years) validity period

      openssl req -new -newkey rsa:2048 -days 3650 -nodes -x509 -keyout /etc/vmware/nsx/host-privkey.pem -out /etc/vmware/nsx/host-cert.pem -config /tmp/tmp-openssl-proxy.cnf -extensions req_ext


      For NSX 4.1.2.5 and higher restarting nsx-proxy restart creates the new cert-key pair:
      
      /etc/init.d/nsx-proxy restart
      
      

   4. Open an SSH session with any NSX Manager node in the cluster as admin. Identify NSX Manager thumbprint
      
      get certificate api thumbprint
      
      
   5. Push the new cert-key pair to the NSX Manager using the root user on the Host or Edge. (The NSX Manager name or IP must be same as used in Step 4)
      
      

      Host
      nsxcli -c push host-certificate <Manager hostname-or-IP> username admin thumbprint <thumbprint from step 4>
      Password for API user: <enter admin password>

      Edge
      su admin -c push host-certificate <Manager hostname-or-IP> username admin thumbprint <thumbprint from step 4>
      Password for API user: <enter admin password>

Additional Information

• Internal error reported when running the NSX cli push host-certificate