Virtual Machines Lose Network Connectivity After NSX-T Upgrade Due to ESXi Host TLS Certificate Trust Failure

Products

• VMware NSX

Issue/Introduction

• Following an NSX-T upgrade, virtual machines (VMs) successfully migrate to a specific ESXi host but lose all network connectivity.
• The VMs remain powered on, but are inaccessible on the network.
• Log analysis of the affected ESXi host reveals the following error in cfgAgent.log: [errorCode="LCP01106"] dfw: Failed to apply rule config to filter nic-[UUID]-eth0-vmware-sfw.2 of vif [VIF-UUID]: invalid argument
• The nsx-opsagent.log may show entries indicating a failure to read the certificate from the NSX Manager: [ConfigViewUpdateConnID] Read certificate for server [NSX-Manager-IP]/[] failed

Environment

VMware NSX

Cause

This issue occurs when the ESXi host loses TLS certificate trust with the NSX Manager or the Control Plane cluster during the upgrade process. Because the host cannot establish a secure connection to the NSX control plane (CCP), it is unable to receive or enforce Distributed Firewall (DFW) policies. When VMs are migrated to this host, the DFW filter application fails, resulting in the loss of network connectivity.

Resolution

To resolve this issue, you must manually refresh the certificate trust between the impacted ESXi host and the NSX Manager.

1. Log in to the affected ESXi host via SSH using root credentials.

2. Execute the following commands to push the host certificate and synchronize certificates with the NSX Manager. Replace <Manager hostname-or-IP> with the actual FQDN or IP of your NSX Manager and provide the correct thumbprint.
   
   nsxcli -c push host-certificate <Manager hostname-or-IP> username admin thumbprint <thumbprint of NSX manager>nsxcli -c push sync-aph-certificates <Manager hostname-or-IP> username admin thumbprint <thumbprint of NSX manager>

    

   Note: You can retrieve the NSX manager thumbprint by executing get certificate api thumbprint on the NSX Manager CLI.

3. Verify that the host has successfully re-established the control plane connection by running the following command:

   nsxcli -c get controllers

   
   The output should confirm a 'connected' status for the controller nodes. Once connected, the host will be able to process DFW policies, and network connectivity for the VMs will be restored.

Additional Information

Preventive Actions

• Prior to placing hosts into maintenance mode for upgrades or maintenance tasks, verify host-to-controller connectivity by executing nsxcli -c get controllers on the ESXi host.
• Ensure that certificate trust validation is included in your post-upgrade health check procedures.