host went to a Not Responding state

Products

• VMware vSphere ESXi

Issue/Introduction

• Host went to "Not Responding" state 
• Errors in hostd logs similar to:Er(163) Hostd[2101754]: [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/vsan:################################/################################/#########.vmx] Could not perform config check (storage not accessible): Fault cause: vim.fault.GenericVmConfigFault
• Lots Messages in vobd log similar to:In(14) vobd[2098051]:  [vmfsCorrelator] 1819831405084us: [esx.problem.vmfs.heartbeat.timedout] ################################ ################################
• vsan logs have message similar to: Er(11) vsand[2101406]: [opID=vsan-################################ statscollector::RetrieveRemoteStats] RetrieveRemoteStats_HighResolutionVsiConnector exception: VMK vmk2 can not connect to host <IP Addres>.
• In /var/log/vmkernel.log you will see the following log lines on a live host
• vmkernel: cpu56:2099594 opID=5bc91eae)Uplink: 18055: vmnic7: set flags 0x59e0e DEVICE_REENABLING 
  vmkernel: cpu56:2099594 opID=5bc91eae)Uplink: 18193: vmnic7: clear flags 0x51e0e DEVICE_REENABLING
  vmkernel: cpu56:2099594 opID=5bc91eae)Uplink: 18055: vmnic4: set flags 0x59e0e DEVICE_REENABLING
   vmkernel: cpu56:2099594 opID=5bc91eae)Uplink: 18193: vmnic4: clear flags 0x51e0e DEVICE_REENABLING

Environment

8.x

Cause

Networking issues on the host causing connectivity issues to storage and host.

Resolution

• Rebooting the host might restart the OS networking stack. 
• Check nic driver/firmware versions and update as necessary.  Contact Hardware vendor for assistance.
• A network connectivity issue between the host and vCenter Server, such as UDP port 902 not open, a routing issue, bad cable, firewall rule, etc.
• Troubleshooting the networking issue is necessary.
  

  Troubleshooting an ESXi host in a "not responding"/"disconnected" state

Additional Information

vSAN Unicast Connectivity Check Fails After Firmware Upgrade Due to Missing Physical Uplinks