NSX Edge upgrade fails in post-upgrade power on task

Products

• VMware Cloud Foundation
• VMware NSX
• VMware Telco Cloud Platform

Issue/Introduction

• The NSX Edge upgrade is performed from VCF.
• Log lines similar to the below are encountered on the NSX Manager in /var/log/upgrade-coordinator/upgrade-coordinator.log

Edge group upgrade status is FAILED: [Edge ###/Edge/nub/VMware-NSX-edge-###.nub post power on task failed on edge TransportNode ########-####-####-####-############: clientType EDGE, target edge fabric node id ########-####-####-####-############, return status Issuing post_power_on failed to complete in time.].

• In vSphere UI, the NSX Edge node VM's interfaces are found in a disconnected state.
• The NSX Edge node upgrade fails in post-upgrade power on task.

Note: The preceding log excerpts are only examples. Date, time, environmental variables and versions may vary depending on your environment.

Environment

VMware NSX
VMware NSX-T Data Center
VMware Cloud Foundation 5.0

TCA 5.1

Cause

Post upgrade, the NSX Edge node's network cards are in a disconnected state in vSphere.

Resolution

This is a known issue impacting VMware NSX.

Workaround:

1. 1. Reconnect the network interfaces of the affected NSX Edge node virtual machine by logging into the vCenter Server.
   2. The NSX Edge node should not be in Maintenance Mode.
   3. Make sure there is a valid uplink attached to the Distributed Virtual Switch.
   4. Retry edge node upgrade either from the SDDC manager UI or via SDDC manager rest API.

NOTE : If the edges are not picking the IP even after reconnecting, we can try reloading the VMXNET3 driver using the command : "modprobe vmxnet3" and restart edge to fix the issue.

Additional Information

Related article:

VMware NSX Edgeのアップグレードは、アップグレード後の電源投入ステップで、Edge仮想マシン上のEdge vNICが切断されることが原因で失敗します。