Alarm For Transport Node Certificate Expiration Approaching

Products

• VMware NSX

Issue/Introduction

Title: Alarm for transport_node_certificate_expiration_approaching
Event ID: transport_node_certificate_expiration_approaching
Alarm Description

• Purpose: Notify User that Transport Node Certificate expiration is approaching in 30 days or less.

• Impact: Transport Node can disconnect from Managers and not connect back again after the certificate expiry.

• Cause: Transport Node certificate expiration is approaching in 30 days or less.

Environment

VMware NSX-T Data Center

Resolution

Steps to Resolve, for 4.1.0 and higher:

• Is a maintenance window required for remediation?
  No
• Steps to resolve:
  Replace the Transport node certificate with a non-expired certificate.
  The expired certificate can be replaced by using below NSX API with json body { "pem_encoded" : "", "private_key":""}.
  Here "pem_encoded" should have new Transport Node Certificate and "private_key" should have new Transport Node private key.
  • NSX API: POST /api/v1/trust-management/certificates/action/replace-host-certificate/{tn_uuid}
• Workaround:
  If the Transport Node certificate has not been replaced within 30 days or the the certificate is already expired.
  Follow below mentioned steps:
  1. Enable ssh for the Transport Node and then ssh to the Transport Node
  2. Delete Transport Node certificate and private key
     CMD: rm -rf /etc/vmware/nsx/host-private.pem /etc/vmware/nsx/host-cert.pem

  3. For NSX-T version >= 4.1.0 to version < 4.1.2.5 run below mentioned commands one by one to generate/replace Host Certificate. Note: skip this step for other versions.
     a)  Create a temporary openssl config file from the existing openssl config

     cat /etc/vmware/nsx/openssl-proxy.cnf > /tmp/tmp-openssl-proxy.cnf

     b) Add the Host UID to the temporary openssl config

     echo "UID = $(grep -o '<uuid>[^<]*' /etc/vmware/nsx/host-cfg.xml | sed 's/<uuid>//')" >> /tmp/tmp-openssl-proxy.cnf

     c) Add extension in the temporary openssl config

     echo -e "[ req_ext ]\nbasicConstraints     = CA:FALSE\nextendedKeyUsage     = clientAuth\nsubjectKeyIdentifier = hash\nauthorityKeyIdentifier = keyid,issuer" >> /tmp/tmp-openssl-proxy.cnf

     d) Generate/Replace Host certificate, where below -days parameter specifies numbers of days certificate will be valid, in the below example it creates the certificate valid for 10 years.

     openssl req -new -newkey rsa:2048 -days 3650 -nodes -x509 -keyout /etc/vmware/nsx/host-privkey.pem -out /etc/vmware/nsx/host-cert.pem -config /tmp/tmp-openssl-proxy.cnf -extensions req_ext

  4.  Restart the NSX proxy. (Restarting the nsx-proxy will create a new self-signed cert-key pair, if Step 3 is not applicable for current version.)
     CMD: /etc/init.d/nsx-proxy restart
  5. Push the new cert-key pair to manager using below nsxcli cmd on Transport Node
     CMD: push host-certificate <hostname-or-ip-address[:port]> username <username> thumbprint <thumbprint>

Additional Information

• Why do we need this?
  This Alarm is raised to notify user that the Transport Node certificate will expire in 30 days or less.