Data Source Error for NSX Manager: Data collection failed due to an error. Please wait for some time. If problem persists, create a support bundle for the Collector VM and contact VMWare support

Products

• VCF Operations for Networks

Issue/Introduction

1. Data Source Error for NSX Manager: Data collection failed due to an error. Please wait for some time. If problem persists, create a support bundle for the Collector VM and contact VMWare support
   
   Refer to error screenshot below:
   
   
   
2. Logs located at location /var/log/arkin/collector/latest.log and /var/log/arkin/collector/latest_error.log  shows multiple entries, as the one below:
   
   

   2025-09-23T16:56:36.664Z ERROR dataprovider.utils.HttpUtils NSXT_IP/FQDN_FirewallConfig_nsxt.metric.exec.numThreads-1 checkStatusAndThrow:41 API /api/v1/firewall/sections//infra/domains/default/security-policies/######-####-###-###-######/rules/stats error response {
       "module_name" : "common-services",
       "error_message" : "The requested URI: /api/v1/firewall/sections/infra/domains/default/security-policies/######-####-###-###-######/rules/stats could not be found.",
       "error_code" : 258
   }
   2025-09-23T16:56:36.664Z WARN dataprovider.southbound.AbstractDPHttpsClusterConnectionEntity NSXT_IP/FQDN_FirewallConfig_nsxt.metric.exec.numThreads-1 increaseFailedCount:189 Increasing the failure count for https://NSXT_IP/FQDN:443
   2025-09-23T16:56:36.664Z WARN common.utils.CommonUtils NSXT_NSXT_IP/FQDN_FirewallConfig_nsxt.metric.exec.numThreads-1 logException:2678 Unable to fetch data from /api/v1/firewall/sections//infra/domains/default/security-policies/######-####-###-###-######/rules/stats
   java.util.concurrent.ExecutionException: com.vnera.dataproviders.core.common.impl.dataprovider.utils.exceptions.HttpException: Could not get response for /api/v1/firewall/sections//infra/domains/default/security-policies//######-####-###-###-######/rules/stats, status 404
           at com.github.rholder.retry.Retryer_ExceptionAttempt.<init>(Retryer.java:174) _[guava-retrying-1.0.5.jar:_]
           at com.github.rholder.retry.Retryer.call(Retryer.java:107) _[guava-retrying-1.0.5.jar:_]
           at com.vnera.dataproviders.core.common.impl.dataprovider.utils.RetryUtils.retryWithExpoBackoff(RetryUtils.java:60) _[dataproviders-0.001-SNAPSHOT.jar:_]
           at com.vnera.dataproviders.core.impl.vmware.nsxt.tasks.NSXTUtils.executeHttpCall(NSXTUtils.java:2573) _[dataproviders-0.001-SNAPSHOT.jar:_]
           at com.vnera.dataproviders.core.impl.vmware.nsxcommon.AbstractClusterDataSourceFetchUtils.fetchData(AbstractClusterDataSourceFetchUtils.java:76) _[dataproviders-0.001-SNAPSHOT.jar:_]
           at com.vnera.dataproviders.core.impl.vmware.nsxt.tasks.NSXTUtils.fetchData(NSXTUtils.java:2597) _[dataproviders-0.001-SNAPSHOT.jar:_]
           at com.vnera.dataproviders.core.impl.vmware.nsxt.tasks.NSXTUtils.getMetricFetchData(NSXTUtils.java:1797) _[dataproviders-0.001-SNAPSHOT.jar:_]
           at com.vnera.dataproviders.core.impl.vmware.nsxt.tasks.metricfetchers.AbstractNSXTMetricFetcher.getFetchData(AbstractNSXTMetricFetcher.java:526) _[dataproviders-0.001-SNAPSHOT.jar:_]
           at com.vnera.dataproviders.core.impl.vmware.nsxt.tasks.metricfetchers.NSXTFirewallConfigMetricFetcher.getResponseForStatistic(NSXTFirewallConfigMetricFetcher.java:50) _[dataproviders-0.001-SNAPSHOT.jar:_]
           at com.vnera.dataproviders.core.impl.vmware.nsxt.tasks.metricfetchers.AbstractNSXTMetricFetcher.processMetricsForEntity(AbstractNSXTMetricFetcher.java:221) _[dataproviders-0.001-SNAPSHOT.jar:_]
           at com.vnera.dataproviders.core.impl.vmware.nsxt.tasks.metricfetchers.AbstractNSXTMetricFetcher.lambda_getPerformanceStatisticsData_1(AbstractNSXTMetricFetcher.java:206) _[dataproviders-0.001-SNAPSHOT.jar:_]
           at com.vnera.common.BatchOperation.lambda_submit_0(BatchOperation.java:95) _[common-0.001-SNAPSHOT.jar:_]
           at com.google.common.util.concurrent.TrustedListenableFutureTask_TrustedFutureInterruptibleTask.runInterruptibly(TrustedListenableFutureTask.java:127) [guava-23.5-jre.jar:_]
           at com.google.common.util.concurrent.InterruptibleTask.run(InterruptibleTask.java:57) [guava-23.5-jre.jar:_]
           at com.google.common.util.concurrent.TrustedListenableFutureTask.run(TrustedListenableFutureTask.java:80) [guava-23.5-jre.jar:_]
           at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) [_:_]
           at java.util.concurrent.ThreadPoolExecutor_Worker.run(ThreadPoolExecutor.java:635) [_:_]
           at java.lang.Thread.run(Thread.java:840) [_:_]
   Caused by: com.vnera.dataproviders.core.common.impl.dataprovider.utils.exceptions.HttpException: Could not get response for /api/v1/firewall/sections//infra/domains/default/security-policies/######-####-###-###-######/rules/stats, status 404
   

Environment

Aria Operations for Networks 6.13.0
Aria Operations for Networks 6.14.0
Aria Operations for Networks 6.14.1

Cause

NSX API used from Aria Operations for Networks shows failed to get response for security policy entity with status 404

This appears to be an intermittent issue but results in the error message for data collection.

Resolution

1. Identify each UUID seen in the logs.
2. Search for the entity UUID for each Security policy in NSXT GUI to validate if they exist.
3. Most of the time they will exist in NSXT UI.
4. If Security policy entity UUID exists and is searchable then follow step 5 onwards
5. Disable the data collection for the NSXT datasource from GUI.
6. Wait for minimum 45- 60 minutes.
7. Enable the data collection from the Collector 
8. Wait for a couple of pooling cycles for the data collector to happen (Approx. 45 minutes).
9. The error on the GUI "Data collection failed due to an error. " should not appear again on Aria Operations for Networks GUI.
10.  Datasource should now be healthy on the GUI and data collector operation resumes as expected.