VM status intermittently changes to "Unavailable" or "Unresponsive" with GuestRpcSendTimedOut errors

Products

• VMware vSphere ESXi

Issue/Introduction

• A Virtual Machine (VM) intermittently transitions from "Active" to "Unavailable" or "Unresponsive" status in vCenter Server.
• The VM may become reachable again only after a guest OS reboot or a hard reset.
• Reviewing the vmware.log file for the affected VM shows repeated entries similar to:
  YYYY-MM-DDTHH:MM:SS.ZZZZ In(05) vmx - GuestRpcSendTimedOut: message to toolbox timed out.
• The ESXi vmkernel.log shows no underlying storage, network, or hardware abnormalities (such as NMP path failures or high latency) during the time of the impact.

Environment

VMware vSphere ESXi 8.x

Cause

The issue is typically isolated to the Guest OS layer. 

The GuestRpcSendTimedOut error indicates that the VMware Tools process (specifically the toolbox or vmtoolsd service) within the guest operating system is failing to respond to communication requests (heartbeats) from the ESXi host.

Resolution

Since the ESXi host is functioning normally, the investigation should focus on the health and performance of the Guest OS.