Host is not responding in the vCenter

Products

• VMware vSphere ESXi

Issue/Introduction

Attempt to access the host using host client fails with message 'Service Unavailable 503'
vim-cmd commands fail to complete with message 'Failed to login: Connection refused: The remote service is not running, OR is overloaded, OR a firewall is rejecting connections'
esxcli commands fail to complete
services.sh restart  command appears to succeed however, the host continues to remains unresponsive

Cause

Underlying storage issues can cause the hostd service to enter a degraded state and eventually, become unresponsive causing the host to show 'not responding' on the vCenter

One of the LUNs that the host had access to, has encountered a hardware failure.

The following failure is repetitively logged in the vmkernel logs of the host as observed in the live vmkernel logging of OOBMS (Out of Bound Management System) - iLO/iDRAC/CIMC
H:0x0 D:0x2 P:0x0 Valid sense data: 0x4 0x8 0x0
0x4 - Hardware Error
08/00 - Logical Unit Communcation Failure

Resolution

To resolve the issue, we may unpresent the offending LUN from the storage array.
Once the errors have stopped appearing in the live vmkernel logging, perform a rescan on all the hosts that had access to the LUN.

Engage your Storage Vendor to investigate the cause of the failure.