"Data collection failed due to an error. Please wait for some time." error message under Accounts and Data Sources page in VCF Operations for Networks

Products

VCF Operations for Networks

Issue/Introduction

The symptom is observed, when under the NSX-T data source, the box "Enable latency metric collection" is checked.

To obtain the information needed to confirm the issue and resolve it, the easiest way is to review collector logs by connecting to the collector VM via SSH.

The collector logs can be found inside the collector VM at the path "/var/log/arkin/collector/"

Inside that directory, typically there will be a series of files. Example below:

collector.STDOUT-####-##-##-03.03.26.log.error
collector.STDOUT-####-##-##-12.01.42.log.error
collector.STDOUT-####-##-##-13.07.13.log
collector.STDOUT-####-##-##-13.25.14.log
collector.STDOUT-####-##-##-13.39.00.log
collector.STDOUT-####-##-##-13.56.15.log
collector.STDOUT-####-##-##-14.12.45.log
collector.STDOUT-####-##-##-14.29.30.log
collector.STDOUT-####-##-##-14.44.10.log
collector.STDOUT-####-##-##-15.00.25.log

In each file name, the "####-##-##" string will be the date in YYYY-MM-DD format, and the number to the right will be the time stamp of the first entry in the file, in HH-MM-SS format (the time zone will be UTC time).

The information you seek will typically be located in the file whose name contains the text string "log.error". Start with the most recent time stamp.

An example command to reveal the information would be (assuming the file list example above):

less collector.STDOUT-####-##-##-12.01.42.log.error | grep -B4 duplicate

A sample output from the above command would have a format like below:

####-##-##T12:18:40.103Z ERROR dataprovider.utils.HttpUtils NSXT_#####################_Config_OpMgr-4 checkStatusAndThrow:41 API /api/v1/service-configs error response {
  "httpStatus" : "BAD_REQUEST",
  "error_code" : 41013,
  "module_name" : "service-config",
  "error_message" : "ServiceConfig precedence value 1 is duplicate for given profile type [LatencyStatProfile]. Specified value is present in ServiceConfig ServiceConfig/########-####-####-###-############"
--
####-##-##T12:48:42.050Z ERROR dataprovider.utils.HttpUtils NSXT_#####################.com_Config_OpMgr-3 checkStatusAndThrow:41 API /api/v1/service-configs error response {
  "httpStatus" : "BAD_REQUEST",
  "error_code" : 41013,
  "module_name" : "service-config",
  "error_message" : "ServiceConfig precedence value 1 is duplicate for given profile type [LatencyStatProfile]. Specified value is present in ServiceConfig ServiceConfig/########-####-####-###-############"

WHERE:

####-##-## is the date of the event in YYYY-MM-DD format (Note: the time follows the letter "T" in HH-MM-SS.MS" format - UTC time zone)
##################### is the IP address or the FQDN of the configured data source
########-####-####-###-############ is the UUID (Universal Unique Identifier) of the object.

NOTES:

The name of the product is VCF Operations for Networks.
It was formerly referred to as "Aria Operations for Networks" and sometimes abbreviated as "AON"
Prior to that, it was referred to as "vRealize Network Insight" and sometimes abbreviated as "vRNI"

Environment

VMware Aria Operations for Networks 6.13
VMware Aria Operations for Networks 6.14

Cause

There is a duplicate Latency Statistic Profile in NSX which causes a latency metric data collection failure in VCF Operations for Networks.

Resolution

The permanent solution before we can enable Latency is to clear the duplicate object "########-####-####-###-############" as obtained above.

If you have not already done so, collect a support bundle for the collector in question, as a baseline.
Uncheck the "Enable latency metric collection" checkbox in the edit datasource page for NSXT on the VCF Operations for Networks GUI
Get all service configs by calling API with text like "https://nsx-ip/api/v1/service-configs/########-####-####-###-############" (GET API) [POSTMAN]
Delete that particular service config again via API by using the corresponding ID:

e.g.: Delete API https://nsx-ip/api/v1/service-configs/########-####-####-###-############
Enable or check the "Enable latency metric collection" checkbox under edit data source page for NSXT on the VCF Operations for Networks GUI.
At this time no errors should be seen and data source should have been submitted successfully.
Observe the environment for possible recurrence, for a period of time which could be up to 48 hours. This period of time can vary with a number of factors, mainly related to the size and scope of the customer environment.
If the issue recurs before the expiration of the 48 hour observation period, please collect a fresh log bundle for the collector in question. If you have already opened a Support case with Broadcom using the instructions at Creating and managing Broadcom support cases then upload both the baseline and the fresh log bundles to the case using the instructions at Uploading files to cases on the Broadcom Support Portal
If you have not yet opened a Support case, do so using the instructions at Creating and managing Broadcom support cases and upload both the baseline and the fresh log bundles to the case using the instructions at Uploading files to cases on the Broadcom Support Portal