Frequent Container Volume and Storage Events observed on vCenter after SSP 5.1 deployment.

Products

• VMware vDefend Firewall with Advanced Threat Prevention
• VMware vDefend Firewall

Issue/Introduction

After upgrading or deploying to SSP 5.1, high number of informational events are being generated in vCenter related to Container Native Storage (CNS) and Kubernetes CSI (Container Storage Interface) operations.

The following event types are observed at a high frequency (every few minutes):

• Update Container Volume
• Delete Virtual Storage Object
• Detach Container Volume
• Attach Virtual Disk
• Retrieve Virtual Storage Object

Environment

SSP 5.1

Cause

The frequent events are a side effect of Druid ingestion tasks. 

Each Druid task, which performs data ingestion, compaction, and reindexing, is created as a K8s job. Each job is self-contained and must request its own storage.

A new Persistent Volume (PV) is dynamically provisioned and mounted onto a worker node every time a K8s job is created. This repeated provisioning and mounting activity triggers the frequent "container volume" and "storage object" events visible in vCenter.

Resolution

A script is provided to switch the Druid ingestion jobs to use ephemeral storage on the worker nodes instead of Persistent Volume Claims (PVCs).

This change will increase disk usage on the worker nodes, therefore it is not recommended if the available disk space is insufficient.

Please Contact Broadcom Support team to run this script.