Aria Operations for Networks - Under Account and data source page, error message "Data collection failed due to an error. Please wait for some time."

Products

VCF Operations for Networks

Issue/Introduction

The symptom is observed, when under the NSX-T data source, the box "Enable latency metric collection" is checked.

NOTES:

The name of the product is VCF Operations for Networks.
It was formerly referred to as "Aria Operations for Networks" and sometimes abbreviated as "AON"
Prior to that, it was referred to as "vRealize Network Insight" and sometimes abbreviated as "vRNI"

Environment

VMware Aria Operations for Networks 6.12
VMware Aria Operations for Networks 6.12.1
VMware Aria Operations for Networks 6.13
VMware Aria Operations for Networks 6.14

Cause

To obtain the information needed in the Resolution section, the easiest way is to review collector logs by connecting to the collector VM via SSH.

The collector logs can be found inside the collector VM at the path "/var/log/arkin/collector/"

Inside that directory, typically there will be a series of files. Example below:

collector.STDOUT-####-##-##-03.03.26.log.error
collector.STDOUT-####-##-##-12.01.42.log.error
collector.STDOUT-####-##-##-13.07.13.log
collector.STDOUT-####-##-##-13.25.14.log
collector.STDOUT-####-##-##-13.39.00.log
collector.STDOUT-####-##-##-13.56.15.log
collector.STDOUT-####-##-##-14.12.45.log
collector.STDOUT-####-##-##-14.29.30.log
collector.STDOUT-####-##-##-14.44.10.log
collector.STDOUT-####-##-##-15.00.25.log

In each file name, the "####-##-##" string will be the date in YYYY-MM-DD format, and the number to the right will be the time stamp of the first entry in the file, in HH-MM-SS format (the time zone will be UTC time).

The information you seek will typically be located in the file whose name contains the text string "log.error". Start with the most recent time stamp.

An example command to reveal the information would be (assuming the file list example above):

less collector.STDOUT-####-##-##-12.01.42.log.error | grep -B4 duplicate

A sample output from the above command would have a format like below:

####-##-##T12:18:40.103Z ERROR dataprovider.utils.HttpUtils NSXT_#####################_Config_OpMgr-4 checkStatusAndThrow:41 API /api/v1/service-configs error response {
  "httpStatus" : "BAD_REQUEST",
  "error_code" : 41013,
  "module_name" : "service-config",
  "error_message" : "ServiceConfig precedence value 1 is duplicate for given profile type [LatencyStatProfile]. Specified value is present in ServiceConfig ServiceConfig/########-####-####-###-############"
--
####-##-##T12:48:42.050Z ERROR dataprovider.utils.HttpUtils NSXT_#####################.com_Config_OpMgr-3 checkStatusAndThrow:41 API /api/v1/service-configs error response {
  "httpStatus" : "BAD_REQUEST",
  "error_code" : 41013,
  "module_name" : "service-config",
  "error_message" : "ServiceConfig precedence value 1 is duplicate for given profile type [LatencyStatProfile]. Specified value is present in ServiceConfig ServiceConfig/########-####-####-###-############"


WHERE:

####-##-## is the date of the event in YYYY-MM-DD format (Note: the time follows the letter "T" in HH-MM-SS.MS" format - UTC time zone)
##################### is the IP address or the FQDN of the configured data source
########-####-####-###-############ is the UUID (Universal Unique Identifier) of the object.

Resolution

The permanent solution before we can enable Latency is to clear the duplicate object "########-####-####-###-############" as obtained above.

If you have not already done so, collect a support bundle for the collector in question, as a baseline.
Disable or uncheck the "Enable latency metric collection" check under edit datasource page for NSXT on the VMware Aria Operations for Networks GUI
Edit the NSXT datasource and uncheck the Latency profile check box
Get all service configs by calling API with text like "https://nsx-ip/api/v1/service-configs/########-####-####-###-############" (GET API) [POSTMAN]
Delete that particular service config again via API by using the corresponding ID:

e.g.: Delete API https://nsx-ip/api/v1/service-configs/########-####-####-###-############
Enable or check the "Enable latency metric collection" checkbox under edit data source page for NSXT on the VMware Aria Operations for Networks GUI.
At this time no errors should be seen and data source should have been submitted successfully.
Observe the environment for possible recurrence, for a period of time which could be up to 48 hours. This period of time can vary with a number of factors, mainly related to the size and scope of the customer environment.
If the issue recurs before the expiration of the 48 hour observation period, please collect a fresh log bundle for the collector in question. If you have already opened a Support case with Broadcom using the instructions at Creating and managing Broadcom support cases then upload both the baseline and the fresh log bundles to the case using the instructions at Uploading files to cases on the Broadcom Support Portal
If you have not yet opened a Support case, do so using the instructions at Creating and managing Broadcom support cases and upload both the baseline and the fresh log bundles to the case using the instructions at Uploading files to cases on the Broadcom Support Portal