Host prepared for NSX lost network connectivity after reboot

Products

• VMware NSX
• VMware vSphere ESXi

Issue/Introduction

• ESXi host was prepared for NSX, and management vmknic was migrated to NSX segment.
• ESXi host is rebooted immediately after migration of management vmknic to the segment.
• After the reboot, ESXi host becomes unreachable.  
• In the logs, you may observe logging similar to sample below:
  var/run/log/vmkernel.log
2024-07-26T08:34:27.941Z cpu253:2098642)WARNING: Tcpip_Vmk: 1771: Could not open a port to ########-####-####-####-########5f7b: Not found
2024-07-26T08:34:27.941Z cpu253:2098642)WARNING: Tcpip_Vmk: 1349: Unable to attach vmk0 to ########-####-####-####-########5f7b [error = 0xbad0003]
2024-07-26T08:34:27.942Z cpu253:2098642)WARNING: Tcpip_Vmk: 1771: Could not open a port to ########-####-####-####-########10b8: Not found
2024-07-26T08:34:27.942Z cpu253:2098642)WARNING: Tcpip_Vmk: 1349: Unable to attach vmk1 to ########-####-####-####-########10b8 [error = 0xbad0003]

Environment

• NSX, or NSX-T Data Center (any version)
• ESXi host's management vmknic is migrated to NSX segment, followed by immediate reboot of the ESXi host. 

Cause

• After the logical switch port is created for the vmknic interface, it is persisted to ESXi host's dvsdata.db in 5 minute intervals.
• If the ESXi host is rebooted before the switchport is persisted in dvsdata.db, this issue will occur.

Resolution

This is a known issue impacting VMware ESXi. This issue will be fixed in future release of VMware ESXi. 

Workaround:

• At least 5 minute interval must be allowed for the ESXi host to persist the switchport in it's internal database before the reboot of the ESXi host. 
• Workaround is to wait 5 minutes after the migration of management (or other vmknic) to nsx backed segment before the host is rebooted.