VMs lost connection to host

Products

• VMware vSphere ESXi

Issue/Introduction

• VMs in vcenter will show disconnected
• Hosts will show disconnected
• You are using NFS storage
• In the /var/log/vobd.log you will see this message:
  

  vobd[######]:  [APDCorrelator] #############us: [esx.problem.storage.apd.start] Device or filesystem with identifier [########-########] has entered the All Paths Down state.

• In the /var/log/vmkernel.log you will see these messages:
  

  [netCorrelator] ############us: [vob.net.vmnic.linkstate.down] vmnic vmnic# linkstate down
  Uplink: vmnic# is down. Affected portgroup: <Port Group Name>. 0 uplinks up. Failed criteria: 128
  
  
  

Environment

7.x

8.x

Resolution

• The host has entered an all paths down state because the vmnic that is used for communicating with the NFS storage is down
• The reason the vmnic is down is showing a criteria 128 which is caused by unplugging the network cable or administratively downing the physical switchport.
• This needs to be investigated from the network infrastructure side to see why the vmnic went down