"System usage has breached the capacity" in VCF Operations for Networks 6.14

Products

VCF Operations for Networks

Issue/Introduction

VCF Operations for Networks displays the following alert within the UI:

"System usage has breached the capacity."

This issue occurs when the observed system flow ingestion exceeds the known capacity limit for the currently deployed cluster size.

You can check your usage and capacity in the UI under Settings > Infrastructure and Support > > System Infrastructure > Platform Capacity

NOTE: VCF Operations for Networks was formerly named Aria Operations for Networks (AON), and prior to that was named vRealize Network Insight (vRNI).

Environment

VCF Operations for Networks 6.14.x

Cause

System flow usage has exceeded the maximum supported capacity of the current cluster configuration.

VCF Operations for Networks requires clusters to be sized at 3, 5, 7, 10 or 15 Platform nodes. Configurations with other quantities of nodes (e.g. 4 nodes) are unsupported and may trigger capacity alerts prematurely as they do not align with standard scaling limits."

References: Tech Docs - Cluster Deployment - Maximum Capacity

Resolution

The resolution involves creating and adding Platform nodes to an existing cluster.

These steps assume an example of an existing cluster of 3 Platform nodes that is to be expanded to 5 Platform nodes.

It is recommended to take notes and screen shots of each step as you go, and paste them into a Word .doc or other such method, in case it is needed in the event of a problem where Broadcom Support is engaged.

STEPS:

Deploy two new Platform nodes from the template matching the currently running version.
Adjust VM specifications for each new node prior to power-on, to be the same as the existing nodes:
- CPUs
- RAM
- Disk
Authenticate to the new nodes as consoleuser and configure the network settings to match the existing management network specifications:
- Static IP addresses
- Subnet
- Gateway
- DNS
- NTP
Execute show-connectivity-status on the new nodes to verify network reachability to the existing cluster nodes.
Complete the setup wizard on each new node to finalize network settings and update the consoleuser password.
- NOTE: It is considered a best practice to make the consoleuser password the same for the new nodes as for the existing nodes.
Create a revert point for each of the 3 existing Platform nodes strictly following the sequential steps outlined in Broadcom KB 314428 - Best practices to shutdown VCF Operations for Networks Clustered deployments to ensure snapshots are taken while nodes are powered off in the correct order to ensure data consistency and avoid a common point of failure during cluster expansion.
Authenticate to the VCF Operations for Networks UI via the Platform 1 node.
Navigate to System Configuration > Cluster Management.
Select the option to expand the cluster, input the IP addresses of the newly deployed Platform nodes, and initiate the expansion.
Verify the new capacity by navigating to Settings > Infrastructure and Support > > System Infrastructure > Platform Capacity and confirm the capacity matches expectation per Tech Docs - System Recommendations and Requirements and the alert has cleared.

Additional Information

If at any time during the process, you feel you are seeing an unexpected result, stop and do not try to resolve it without Broadcom assistance.

You can open a support case for the VCF Operations for Networks team using the instructions at KB 142884 - Creating and managing Broadcom cases