The hvc service temporarily stops when another vCenter Server service consumes excessive memory

Products

VMware vCenter Server 8.0 VMware vCenter Server

Issue/Introduction

The health status of the hvc service may temporarily change from green to red or unknown before returning to green.

The hvc Java process may record a native memory allocation failure similar to the following.

YYYY-MM-DDTHH:MM:SS OpenJDK 64-Bit Server VM warning: INFO: os::commit_memory failed; error='Cannot allocate memory'

During the same period, another vCenter Server service may be consuming excessive memory. In the observed case, the vc-ws1a-broker accesscontrol process repeatedly performed Full GC without reclaiming Java heap memory.

YYYY-MM-DDTHH:MM:SS GC Pause Full Allocation Failure 121M->121M

Environment

VMware vCenter Server 8.0

Cause

The available memory at the vCenter Server operating system level was exhausted.

The logs suggest that excessive memory consumption by another vCenter Server service affected the hvc service. In the observed case, the vc-ws1a-broker accesscontrol process was under sustained memory pressure.

Multiple known issues may cause increased memory consumption by vc-ws1a-broker.

Resolution

Review the Broadcom Knowledge Base articles listed below.

Identify the applicable article based on the vCenter Server version, service logs, GC logs, CPU utilization, and swap utilization. Follow the Resolution section of the applicable article.

The following articles describe known memory exhaustion conditions involving vc-ws1a-broker.

Login to vCenter Server with External Identity Providers for SSO account stops working (378474)

High CPU utilization on vCenter Server 8.0 (367603)

vCenter services experiencing OutOfMemoryError: Java heap space, causing the swap memory to be 100% utilized and due to which vCenter backup fails with Error: database or disk is full (396939)

vCenter Server alarm vc-ws1a-broker status changed from green to unknown caused by OutOfMemoryError (420312)

vc-ws1a-broker health status becomes unknown due to java.lang.OutOfMemoryError in the accesscontrol service (437627)