Intelligence service has been disabled on the ESX TNs post NSX upgrade

Products

• VMware NSX

Issue/Introduction

NSX upgrade from previous releases such as 4.0.1 to 4.1.2/ 3.2.3 to 4.1.2 intermittently leads to a few TNs having data collection disabled on it or in some cases, host/clusters not appearing on the NSX UI.

Note: This issue is not seen on upgrades from 4.1.1 to 4.1.2.

Environment

• VMware NSX-T
• VMware NSX-T Data Center 3.x
• VMware NSX-T Data Center 4.x
• VMware NSX-T Data Center

Cause

This issue is primarily caused by two threads racing to modify the same database object. Sometimes the race conditions subsequently resolve, but most times the retries are exhausted.

Resolution

Fixed in 4.2.0 - Fix is to make sure the race conditions are completely avoided.

Workaround:

• If only few TNs are not reporting flows, then toggle data collection on the UI on hosts which are behaving incorrectly. This should reset the configuration.
• If no hosts or clusters are visible on the UI, then run the following APIs to resolve the issue.
  
  1. Send get request to fetch cluster Id
  
  GET https://{{mgr_ip}}/policy/api/v1/infra/sites/napp/registration
  
  2. Set is_intelligence_enabled to false by sending below patch request
  
  PATCH https://{{mgr_ip}}/policy/api/v1/infra/sites/napp/registration/{{cluster-id}}
{
    "cluster_id": "exxxxxx2-exxe-yxd0-b5ad-cabxxb48erqwf8",
    "is_intelligence_enabled": false, <------------ set to false.
    "id": "exxxxxx2-exxe-yxd0-b5ad-cabxxb48erqwf8"

}
  
  3. Re-set is_intelligence_enabled to true by sending the below patch request
  
  PATCH https://{{mgr_ip}}/policy/api/v1/infra/sites/napp/registration/{{cluster-id}}
{
    "cluster_id": "exxxxxx2-exxe-yxd0-b5ad-cabxxb48erqwf8",
    "is_intelligence_enabled": true, <------------------ set to true.
    "id": "exxxxxx2-exxe-yxd0-b5ad-cabxxb48erqwf8"

}

Additional Information

Impact/Risks: No flows are reported from a few hosts. Unable to toggle data collection via UI.