ESXi host Not Responding: InsufficientMemoryResourcesFault and InvalidLogin

Products

• VMware vSphere ESXi

Issue/Introduction

In a VMware vSphere environment, one or more ESXi hosts unexpectedly drop into a Not Responding or Disconnected state within the vCenter Server inventory. While the management control plane is completely disrupted, virtual machines residing on the affected host continue running normally on the data plane with zero workload interruption.

The following symptoms are observed:

• The affected ESXi host displays a grayed-out Not Responding status in vCenter Server.
• Core management actions (such as provisioning, configuration changes, or performance charting) fail or time out.
• Automated cluster services like vSphere HA or DRS may generate configuration synchronization alarms for the affected host.

Log Signature Patterns

Reviewing the /var/run/log/hostd.log on the affected ESXi host reveals a cascade of high-frequency failed login attempts immediately followed by management heap exhaustion errors:

Hostd[XXXXXXXX]: [Originator@6876 sub=AdapterServer opID=HB-SpecSync-host-<HOST_ID>@0-xxxxxxxx-xx-xxxx sid=xxxxxxxx user=vpxuser:<no user>] AdapterServer caught exception; <<xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx, <TCP '127.0.0.1 : 8307'>, <TCP '127.0.0.1 : xxxxx'>>, ha-root-pool, vim.ResourcePool.updateChildResourceConfiguration, <vim.version.v8_0_3_0, internal, 8.0.3.0>, [N11HostdCommon18VmomiAdapterServer19ActivationResponderE:0x000000c6c258af98]>, N7Hostsvc34HaInsufficientMemoryResourcesFault9ExceptionE(Fault cause: vim.fault.InsufficientMemoryResourcesFault

Hostd[XXXXXXXX]: --> (vim.fault.InsufficientMemoryResourcesFault) {Hostd[XXXXXXXX]: -->        message = "Group vm.<RESOURCE_ID>: Requested memory minLimit ##### KB insufficient to support effective reservation ##### KB 

Hostd[XXXXXXXX]: [Originator@6876 sub=Solo.Vmomi] Arg userName: Db(167) Hostd[XXXXXXXX]: --> "<SERVICE_ACCOUNT>@<DOMAIN>.local" Db(167) Hostd[XXXXXXXX]: [Originator@6876 sub=Solo.Vmomi] Arg password: Db(167) Hostd[XXXXXXXX]: --> (not shown)

Db(167) Hostd[XXXXXXXX]: --> Db(167) Hostd[XXXXXXXX]: [Originator@6876 sub=Solo.Vmomi] Arg locale Db(167) Hostd[XXXXXXXX]: --> (null)In(166) Hostd[XXXXXXXX]: [Originator@6876 sub=Solo.Vmomi] Throw vim.fault.InvalidLogin

Cause

This issue is caused by a cascading resource starvation event within the primary host management daemon (hostd). The failure mechanism develops through two compounding factors:

1. High-Frequency Authentication Storm (InvalidLogin): An external monitoring integration, data-collector, or automated script architecture continuously targets the ESXi SDK endpoint using stale, expired, or invalid credentials (<SERVICE_ACCOUNT>@<DOMAIN>.local). Every incoming connection forces hostd to allocate tracking threads and short-lived memory tracking structures within its service heap. Because these queries fail at a high velocity with a fault.InvalidLogin fault, the overhead associated with failed connection handling outpaces the agent's internal garbage collection cycles, creating severe worker thread and memory starvation.
2. Resource Scheduler Exhaustion (InsufficientMemoryResourcesFault): As a direct consequence of the memory leaks induced by the login storm, the host's internal resource scheduler runs out of available operating heap space. During standard SpecSync cluster heartbeats, the management engine attempts to synchronize configuration groups. When a specific management group requires an explicit memory allocation (e.g., 224,668 KB) that exceeds the host's depleted threshold limit (e.g., 148,480 KB), the kernel blocks the transaction and records a fault.InsufficientMemoryResourcesFault.

Ultimately, hostd becomes entirely unresponsive to external network sockets, causing vCenter Server to lose its heartbeat connection and report the host as Not Responding.

Resolution

To resolve this issue and permanently restore control plane stability, you must isolate and stop the rogue external authentication stream before flushing the exhausted hypervisor management heap pools.

Step 1: Terminate the Rogue Authentication Loop

1. Review the host logs or your centralized network security logs to locate the source IP address delivering the automated web service requests with the account <SERVICE_ACCOUNT>@<DOMAIN>.local.
2. Access the external application, monitoring agent, or script manager hosting this integration.
3. Temporarily stop the polling service, or update its configuration with active, valid credentials to halt the continuous wave of bad payloads.

Step 2: Flush and Reallocate Host Management Memory Pools

Once the external traffic storm has been stopped, clear the leaked thread handles and memory spaces on the affected ESXi host by cycling the local management agents.

1. Establish an administrative Secure Shell (SSH) session to the affected ESXi host using root credentials.
2. Execute the following commands to safely restart the management daemons:

     /etc/init.d/hostd restart

     /etc/init.d/vpxa restart

Operational Impact Notice: This process safely flushes all hung connection tracking states from the active system memory plane. The ESXi host will briefly disconnect from the vCenter Server inventory for less than 60 seconds before reconnecting automatically. This operation impacts only the management plane; it does not interrupt, restart, or degrade production virtual machine workloads or virtual networks.

Step 3: Post-Remediation Verification

1. Confirm that the ESXi host automatically recovers to a stable, healthy Connected status within the vCenter Server inventory view.
2. Monitor /var/run/log/hostd.log for several minutes to verify that new fault.InvalidLogin occurrences have completely ceased.