Flow metrics stop collecting for NSX-T Edge Nodes in VCF Operations for Networks

Products

• VCF Operations for Networks

Issue/Introduction

• Flow data for specific NSX-T Edge nodes or Transport nodes is missing from the VCF Operations for Networks (formerly Aria Operations for Networks) UI.
• The NSX-T data source shows a "Green" or "Success" status, but recent flow metrics are unavailable.
• Running the UI search query NSX-T Transport Node where Vendor ID is set returns  all the entities and then filtered by Edge to identify the specific NSX-T Edge nodes or Transport nodes not in inventory. 
• In the Collector logs (/var/log/arkin/collector/collector.STDOUT), you see indications that inventory tasks are starting but not completing.

Environment

VCF Operations for Networks 9.0.x

Cause

This issue is typically caused by a thread-locking condition within the Collector's inventory synchronization service. When the nsxt.TransportNode.inventory task becomes thread-locked, the appliance cannot complete the mapping of incoming IPFIX (UDP 2055) flow packets to the corresponding inventory objects.

Resolution

To resolve this issue, you must force a refresh of the inventory synchronization to clear the locked threads.

Step 1: Verify Connectivity

Before performing a reset, ensure that flows are reaching the collector:

1. Log into the Collector CLI via SSH.
2. Run a tcpdump to verify incoming IPFIX traffic:
   
   

   tcpdump -i eth0 port 2055

    

Step 2: Toggle Data Collection

If packets are arriving but not displaying:

1. Navigate to Settings > Accounts and Data Sources.
2. Locate the affected NSX-T Manager data source.
3. Click the Edit icon and toggle the collection status to Disabled/Off.
4. Wait for approximately 5–10 minutes to allow existing threads to time out and clear.
5. Toggle the collection status back to Enabled/On.
6. Wait 15–30 minutes for the inventory sync to complete and flows to begin appearing.

Step 3: Collect Debug Data (If Issue Persists)

If the flows stop again after a period of time:

1. Do not reboot the appliance or restart services immediately.
2. Generate a  Thread Dumps from the collector node. See command below:
   

   jps -l (identify the collectorMain process id from this command)
   jstack -l <PROCESS_ID>  >& jst.output

   See steps below:
   

   ubuntu@aria-networks-collector :~ $ jps -l
   3409 /home/ubuntu/build-target/launcher/launcher-0.001-SNAPSHOT.jar
   3170 com.vnera. latency. NetopaCollector
   10308 com. vnera.collector.CollectorMain
   2998 com.vnera.telemetry.SwitchTelemetryServer
   21064 jdk.jcmd/sun.tools.jps.Jps
   4425 com.vnera.ipfix. IpfixProcessor
   3007 com.vnera.telemetry.NSXMetricServer
   ubuntu@aria-networks-collector :~ $ jstack -l 10308 >& jst.output

    

3. Move the generated jst.output file to /home/support/ from /home/ubuntu
   
   See commands below:
   

   ubuntu@aria-networks-collector:~$ sudo cp jst.output /home/support/
   ubuntu@aria-networks-collector:~$ exit
   logout
   support@aria-networks-collector:~$ ls -lrth
   total 188K
   -rw-r----- 1 root root 185K Jun 23 21:20 jst.output
   support@aria-networks-collector:~$ sudo chown support:support jst.output
   support@aria-networks-collector:~$ ls -lrth
   total 188K
   -rw-r----- 1 support support 185K Jun 23 21:20 jst.output
   support@aria-networks-collector:~$
   

4. Winscp the above file to local system
5. Generate support bundle for the collector in question
6. Open a support case with Broadcom Support using the directions at KB 142884 - Creating and managing Broadcom cases to review support bundle and thread dump