Data Collection Failure for NSX data source due to Latency Profile deletion error

Products

VCF Operations for Networks VMware NSX

Issue/Introduction

NSX Manager data sources encounter the following error message during data collection in VCF Operations for Networks:

"Data collection failed due to an error. Please wait..."

The following API error response related to the latency profile was identified in the AON/vRNI Collector log located at /var/log/arkin/collector where you will find entries with the following error:

"An object with the same path=[/infra/latency-profiles/vRNI-Latency_Stat_Profile_##########==] is marked for deletion. Either use another path or wait for the purge cycle (max 5 minutes) for permanent removal of the object."

as seen in the following examples:

YYYY-MM-DDTHH:MM:SS.SSSZ ERROR dataprovider.utils.HttpUtils NSXT_<NSX Manager FQDN>_Config_OpMgr-0 checkStatusAndThrow:41 API /policy/api/v1/infra/latency-profiles/vRNI-Latency_Stat_Profile_##########== error response {
"httpStatus" : "BAD_REQUEST",
"error_code" : 500045,
"module_name" : "Policy",
"error_message" : "An object with the same path=[/infra/latency-profiles/vRNI-Latency_Stat_Profile_##########==] is marked for deletion. Either use another path or wait for the purge cycle (max 5 minutes) for permanent removal of the object."
}
YYYY-MM-DDTHH:MM:SS.SSSZ WARN common.utils.CommonUtils NSXT_<NSX Manager FQDN>_Config_OpMgr-0 logException:2555 Unable to fetch data from /policy/api/v1/infra/latency-profiles/vRNI-Latency_Stat_Profile_##########==
java.util.concurrent.ExecutionException: com.vnera.dataproviders.core.common.impl.dataprovider.utils.exceptions.HttpException: Could not get response for /policy/api/v1/infra/latency-profiles/vRNI-Latency_Stat_Profile_##########==, status 400
at com.github.rholder.retry.Retryer_ExceptionAttempt.(Retryer.java:174) _[guava-retrying-1.0.5.jar:_]
at com.github.rholder.retry.Retryer.call(Retryer.java:107) _[guava-retrying-1.0.5.jar:_]
at com.vnera.dataproviders.core.common.impl.dataprovider.utils.RetryUtils.retryWithExpoBackoff(RetryUtils.java:60) _[dataproviders-0.001-SNAPSHOT.jar:_]

You can may also run the following GET request to the NSX Manager Node to retrieve latency profiles and it returns an empty response:

GET https://<manager-ip>/policy/api/v1/infra/latency-profiles

However, there are outputs from the following GET request which indicates the presence of stale entries in NSX Manager Node for the relaized entities:

GET https://<manager-ip>/policy/api/v1/infra/realized-state/realized-entities?intent_path=/infra/latency-profiles/vRNI-Latency_Stat_Profile_##########==

Environment

VMware Aria Operations for Networks

VMware NSX

Cause

This happened because of the presence of stale latency profile entries existing in the NSX Manager Corfu DB

Resolution

This is a known issue and resolved in NSX 4.2.1. If you encounter this issue, contact Broadcom Support: Broadcom Support Portal

Additional Information

Fixed Issue 3394029: The Policy Latency Profile is stuck deleting.

When deleting the Policy Latency profile, it is stuck deleting due to the Corfu update conflict.

https://techdocs.broadcom.com/us/en/vmware-cis/nsx/vmware-nsx/4-2/release-notes/vmware-nsx-421-release-notes.html