NSX Traceflow inaccessible, Error" Traceflow request timeout. Please retry"

Products

• VMware NSX

Issue/Introduction

The Customer cannot use traceflow function. The traceflow function cannot be initiated with the error "Traceflow request timeout. Please retry"

Relevant logs to look: 

On NSX manager root mode, grep -i "Evicted TraceflowModel" /var/log/proton/nsxapi.log (or the compressed nsxapi logs)

2024-10-14T08:00:35.162Z INFO providerTaskExecutor-14 TraceflowCacheManager 81152 MONITORING [nsx@6876 comp="nsx-manager" level="INFO" reqId="58######-f0##-4f#3-8##f-fc72########" subcomp="manager" username="nsx_policy"] Evicted TraceflowModel{traceflowModelMsg=null, traceflowModelMsgOrBuilder=managed_resource {
  display_name: "0f2#####-2##6-####-a##7-adf4########"
}
logical_port_id {
  left: 122############72658
  right: 1222############6983
}
---------------
2024-10-14T08:07:10.047Z INFO org.corfudb.runtime.collections.streaming.StreamPollingScheduler-worker-2 TraceflowStreamListener 81152 MONITORING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] TraceflowStreamListener, get delete operation
2024-10-14T08:07:10.047Z INFO org.corfudb.runtime.collections.streaming.StreamPollingScheduler-worker-2 TraceflowCacheManager 81152 MONITORING [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Deleted traceflow TraceflowModel/ from local cache

Environment

VMware NSX 4.x

Cause

The traceflow resources on the non-source MP nodes have not been correctly cleared.

Resolution

This issue is resolved in VMware NSX 4.2, available at Broadcom downloads.

If you are having difficulty finding and downloading software, please review the Download Broadcom products and software KB.

 

Workaround:

Restart all NSX Manager nodes.

or

Clear the leaked traceflow resources on the non source MP node.

1. Remove all the traceflows via the policy API before user trying to initiate a new traceflow.
   1. Get all the traceflows via RESTAPI

      GET https://<mgr-ip>/policy/api/v1/infra/traceflows. 

      User can get the each id of the traceflows from the response of the RESTAPI.
   2. Delete all the traceflows listed in the previous RESTAPI

      DELETE https://<mgr-ip>/policy/api/v1/infra/traceflows/<traceflow-id>.

2. Now, the user can try the traceflow via UI normally.

Additional Information

Maintenance Window Guidelines for Customer and GSS:

No change is required from the customer side. The user must update the version with the fix or try the workaround.

 

If the issue does not match this KB, kindly validate this where if the issue is more specific to NAT rules.

NSX - Traceflow fails with Error "Traceflow request timeout. Please retry."