ESXi host is in Non-Responding or Disconnected state in vCenter with missing log segments in host level logs.

Products

• VMware vSphere ESXi

Issue/Introduction

• An ESXi host goes into a non-responding or disconnected state in vCenter.
  
  
• When investigating the ESXi logs, significant time gaps are noted:
  

  
/var/run/log/hostd.log:
YYYY-MM-DDTHH:MM:SS Er(163) Hostd[#######]: [Originator@6876 sub=VMkernelStatsProvider(##############)] GetKernelStatValues: Detected error while retrieving stats: VSINode(2437): Not supported (status=##########)
>> timestamp gap is observed
  YYYY-MM-DDTHH:MM:SS In(166) Hostd[#######]: - time the service was last started YYYY-MM-DDTHH:MM:SS, Section for VMware ESX, pid=#######, version=<version>, build=<build>, option=Release

  /var/run/log/vmkernel.log:
YYYY-MM-DDTHH:MM:SS In(182) vmkernel: cpu##:#######)Unmap6: 10568: [DATASTORE]: Unmap capability 0 OP i: 0 offset: 65536 endROffset: 589824 Acquired: TRUE
YYYY-MM-DDTHH:MM:SS In(182) vmkernel: VMB: 65: Reserved 4 MPNs starting @ 0x4c4

• In some scenarios, post-reboot logs suggest a hardware issue:
  
  /var/run/log/vobd.log:
YYYY-MM-DDTHH:MM:SS In(14) vobd[########]  [GenericCorrelator] 62404485us: [vob.user.host.boot] Host has booted.
...
YYYY-MM-DDTHH:MM:SS In(14) vobd[########]  [GenericCorrelator] 62686031us: [vob.user.host.poweroff.reason.unavailable] The host is being powered off. The poweroff was not the result of a kernel error, deliberate reboot, or shut down. This could indicate a hardware issue. Hardware may reboot abruptly due to power outages, faulty components, and heating issues. To investigate further, engage the hardware vendor.

Cause

With no other significant indicators present. The above logging indicates environmental issues such as:

• Power loss
• Firmware issue
• Hardware issue 

Resolution

Proceed to reboot the host and monitor for further outages or engage the hardware support.