High Disk Utilization is seen on VCF Operations for Networks GUI

Products

• VCF Operations for Networks

Issue/Introduction

High Disk Utilization is seen on VCF Operations for Networks GUI

This is seen on all the platform Clustered Nodes. Refer to error screenshots as below:

/var/ partition shows 89% when executed below  command

./run_all.sh df -h

Collector logs at location /var/log/arkin/collector  below error are seen:

2025-05-28T17:26:42.271Z INFO nsxt.tasks.ManageNSXTLatencyTask NSXT_<NSXT IPAddress/FQDN>_Config_OpMgr-0 createNSGroupForTNs:613 New Child NSGroup added, id /infra/domains/default/groups/vRNI-Node_Group_Profile_TN_829###ef-###-###-8453-#######362, member count 40

2025-05-28T17:26:42.326Z ERROR dataprovider.utils.HttpUtils NSXT_<NSXT IP Address/FQDN>_Config_OpMgr-0 checkCodeAndThrow:53 Could not get response for /policy/api/v1/infra/domains/default/groups/vRNI-Node_Group_Profile_RFF####==, status 400
2025-05-28T17:26:42.326Z ERROR dataprovider.utils.HttpUtils NSXT_<NSXT IPAddress/FQDN>_Config_OpMgr-0 checkStatusAndThrow:41 API /policy/api/v1/infra/domains/default/groups/vRNI-Node_Group_Profile_RFF####== error response {
  "httpStatus" : "BAD_REQUEST",
  "error_code" : 500127,
  "module_name" : "Policy",
  "error_message" : "Cannot create an object with path=[/infra/domains/default/groups/vRNI-Node_Group_Profile_RFF####==] as it already exists."
}

Login to NSXT GUI and Navigate to Inventory>Groups

Search with vRNI or vRNI Node_Group_Profile_TN  you should see a huge number of profiles created. 

Refer to screenshots below:

Environment

VCF Operations for Networks

Cause

An investigation of collector logs revealed that improper cleanup procedures resulted in the removal of vRNI\_Groups from the vRNI KV Store, while corresponding entries were not deleted from NSX-T.
Consequently, during latency enablement, while TN_Groups were successfully created, an exception was encountered during the creation of vRNI_Group, indicating a pre-existing group.

During this time the Watcher task operates continuously in the background, generating a substantial number of TN_Groups. Because of which we saw a high number of TN_Groups.

Resolution

VCF Operation for Networks Product team is aware of this issue and is working on a fix for this. 

To workaround this issue this issue we need to:

1. Disable latency metric from VCF Operations for Networks GUI

2. Clean up the stale entries for the profile vRNI-Node_Group_Profile_TN_* on the NSXT GUI.

Open a support case with Broadcom Support to review your VCF Operations for Networks deployment. For more information, see Creating and managing Broadcom support cases.