Virtual machine vNIC does not connect to port and loses network connectivity

Products

• VMware NSX

Issue/Introduction

• When an environment suffers a complete network outage due to the physical network failing, most virtual machines will return to service and connect to the network once the physical network has been repaired and brought back up.
• A few virtual machines may not and will have ports that are blocked.
• This article will focus on ports that are in a blocked state and connected to NSX managed networks.
  
  
• This is an example of a port attachment for an NSX VIF seen in the hostd logs of the ESX host.
  Hostd[2128731]: [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/vsan:################-################/#########-####-####-####-############/VM-NAME.vmx opID=host-######:##:########-#-#### sid=#######user=vpxuser] ProcessOpaqueNetChanges - op:[Register] new:{ [4000->####-####-####-####-##########:nsx.LogicalSwitch:#####-####-###-####-####]} old:{} same:{}
 In(166) Hostd[2128731]: [Originator@6876 sub=Hostsvc opID=host-####:##:####-####-##### sid=###### user=vpxuser] AttachPort requested for vif: [######-####-###-###-####], network: [####-####-####-####-####:nsx.LogicalSwitch], vmx: [/vmfs/volumes/vsan:####-####/####-####-####-####-####/VM-NAME.vmx]
• This is what an example of the VIF attaching to the ESX host managed port looks like.
  In(182) vmkernel: cpu37:11556108)swapobj: SwapObjDumpVmklinkResponse:1493: {ID:0x28c; type:CREATE; ret:{status:bad0005; open:{lock:; dev:s/volumes/vsan:1vsan:xxxxxxxxxxxxxxxx-####/####-####-####-####-####/VM-NAME-####.v$
In(182) vmkernel: cpu37:11556108)World: vm 11556167: 6916: Starting world vmm0:VM-NAME of type 8
In(182) vmkernel: cpu48:11556108)Net: 2241: connected VM-NAME.eth0 eth0 to vDS, portID 0x400005d
  
  
• The ESXi host knows the port as portID 0x500005d and NSX know this port as VIF ####-####-####-####-####. The VIF Id will be used to identify the port connection of the VM's vNIC for NSX to manage. The port ID is used by vCenter to manage the port.
• The creation of the VIF identifier occurs when the host is used as a transport node and the VM is connected to the NSX overlay network.
• This process depends on the nsx-opsagent(ESXi) communication with the NSX managers. If this communication link is not up when these changes are made the port will be blocked.
  There is no retry functionality for these calls.

Environment

VMware NSX

Cause

• Network connectivity is not consistent between NSX and vSphere.
• NSX-opsagent is unstable.
• Virtual machines connect to virtual vswitches via virtual ports.

Resolution

• Ensure that the network connectivity is healthy between NSX and vSphere components.
• Check health of ESXi hosts.
• Disconnect the VM's vNIC and then reconnect it from within the settings of the VM.
  
• The second remediation method is vMotion the VM to another host.