ESXi host not responding in vCenter due to LUN Hardware Failure - VMware vSphere ESXi
search cancel

ESXi host not responding in vCenter due to LUN Hardware Failure - VMware vSphere ESXi

book

Article ID: 396065

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

  • Attempting to access the host using the host client fails with the message: Service Unavailable 503
  • vim-cmd commands fail to complete with: Failed to login: Connection refused: The remote service is not running, OR is overloaded, OR a firewall is rejecting connections
  • esxcli commands fail to complete.
  • services.sh restart appears to succeed, but the host remains unresponsive.

Environment

VMware vSphere ESXi

Cause

Underlying storage issues can cause the hostd service to enter a degraded state, eventually becoming unresponsive and causing the host to show as "not responding" in vCenter. Specifically, one of the LUNs the host has access to may have encountered a hardware failure.

The following failure is repetitively logged in the host's vmkernel logs (observable via Out-of-Band Management Systems like iLO/iDRAC/CIMC):

  • H:0x0 D:0x2 P:0x0 Valid sense data: 0x4 0x8 0x0
    • 0x4: Hardware Error
    • 08/00: Logical Unit Communication Failure

Resolution

  1. Unpresent the LUN: Unpresent the offending LUN from the storage array.
  2. Verify Logs: Ensure the errors have stopped appearing in the live vmkernel logging.
  3. Rescan: Perform an adapter and VMFS rescan on all hosts that had access to the LUN:
    • esxcli storage core adapter rescan --all
    • vmkfstools -V
  4. Engage Vendor: Engage your storage vendor to investigate the root cause of the hardware failure.

Important Note: If hosts become unresponsive after unpresenting the LUN, ensure they are running on ESXi 7.0 U3 P10, 8.0 U2, or later. For environments requiring a fix for LUN unmapping handling, consult your release notes for the appropriate patch levels.