Host reboot is stuck at “Initializing Init_VMKernelShutdownHelper: (11/39) Reboot_RunHandlersScsiEarly ...” for several hours

Products

• VMware vSphere ESXi
• VMware vSphere ESXi 8.0
• VMware vSphere ESX 7.x

Issue/Introduction

• The console displayed “Initializing Init_VMKernelShutdownHelper: (11/39) Reboot_RunHandlersScsiEarly ...” and then stuck for several hours.

• The following events were observed in the vmkernel.log immediately before the reboot:

YYYY-MM-DDTHH:MM:SS.SSSZ cpu46:2098243)NMP: nmp_ThrottleLogForDevice:3867: Cmd 0x0 (0x45d901a213c8, 0) to dev "naa.600#############################" on path "vmhba#:C#:T#:L#" Failed:
YYYY-MM-DDTHH:MM:SS.SSSZ cpu46:2098243)NMP: nmp_ThrottleLogForDevice:3875: H:0x1 D:0x0 P:0x0 . Act:NONE. cmdId.initiator=0x4538d141bc18 CmdSN 0x0
YYYY-MM-DDTHH:MM:SS.SSSZ cpu46:2098243)NMP: nmp_ThrottleLogForDevice:3867: Cmd 0x0 (0x45d901a213c8, 0) to dev "naa.600#############################" on path "vmhba#:C#:T#:L#" Failed:
YYYY-MM-DDTHH:MM:SS.SSSZ cpu46:2098243)NMP: nmp_ThrottleLogForDevice:3875: H:0x0 D:0x8 P:0x0 . Act:NONE. cmdId.initiator=0x4538d141bc18 CmdSN 0x0

Environment

VMware vSphere ESXi 7.0

VMware vSphere ESXi 8.0

Cause

Storage I/O issues caused by failures in storage devices or SAN may result in extended reboot times.

For example, status codes like the following generally indicate a hardware-related issue.

• H:0x1 D:0x0 P:0x0
  Understanding SCSI host-side NMP errors/conditions in ESXi
  VMK_SCSI_HOST_NO_CONNECT
  This status is returned if the connection is lost to the LUN. This can occur if the LUN is no longer visible to the host from the array side or if the physical connection to the array has been removed.
  
  
• H:0x0 D:0x8 P:0x0
  Understanding SCSI Target NMP Errors and Conditions in ESXi
  VMK_SCSI_DEVICE_BUSY
  This status is returned when a LUN cannot accept SCSI commands at the moment. As this should be a temporary condition, the command is tried again.
  
  

Resolution

• Please contact the hardware vendor and request their support.
• If issues persist even after resolving storage I/O errors, serial port output during the reboot process can be useful for troubleshooting.

Enabling serial-line logging for ESXi