Primary node crashes intermittently - VCF Operations for Logs 9.0.x

Products

• VCF Operations

Issue/Introduction

• The VCF Operations for Logs UI may be unavailable.
• There may be 100% CPU spikes on the Primary node.
• Service restarts of the Primary node can be seen on the Cluster status page.

Environment

VCF Operations for Logs 9.0.x

Cause

This behavior occurs in VCF Ops 9.0.x deployments after VC/ESX log collection has been configured through the VCF Operations -> Log Management -> Log Collection UI.

When log collection is established in this way, VCF Operations for Logs makes repeated calls to VC and ESX to maintain the log collection configuration. The primary node restarts are due to the incorrect closing of TLS connections, which eventually causes OutOfMemory errors.

Resolution

When the OutOfMemory error occurs, the primary node will restart and will recover. An email notification may be received. During the restart the UI will not be available. After the restart completes, the system will operate as normal.

Broadcom is aware of this issue in VCF Operations for Logs 9.0.x and is currently developing a permanent fix. Subscribe to this article to receive notifications as updates become available.