NDR Install Stuck at 68% and Eventually Fails when Internet Connectivity is not Available
Article ID: 394965
Updated On:
Products
Issue/Introduction
In cases where NDR is being installed without network connectivity, NDR installation may get stuck at 68% progress, and eventually fail.
Environment
Security Services Platform 5.0
Cause
Cause 1)
During installation of NDR, a component fetches updated threat metadata from VMware cloud services to initialize the NDR feature. If internet connectivity is not available at this time, the component will retry multiple times, and the installation process will remain stuck at 68%. Eventually, a timeout will be reached and the installation will fail with a generic error.
The specific issue can be confirmed by examining the logs of the nsx-metadata-service-load-feed-cronjob, which will show errors in reaching https://api.prod.nsxti.vmware.com. An example log when encountering this issue is shown below. Note that in this case the specific failure is a name resolution error, but the details of the connectivity error may vary depending on the environment.
Error while connecting to NTICS API server. It might be due to a temporary network or client/server side issue. Retrying one more time... - Failed to communicate with server: HTTPSConnectionPool(host='api.prod.nsxti.vmware.com', port=443): Max retries exceeded with url: /2.0/auth/register (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at
0x73993a420970>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))
Cause 2)
NDR activation failed:
Both NDR pods are stuck in ERROR state.
ERROR deployment/chart.go:135 Failed to install helm chart {"error": "client rate limiter Wait returned an error: context deadline exceeded"}
WARN deployment/chart.go:137 Chart installation failed. {"chart": "nsx-metadata-service-evaluation", "error": "client rate limiter Wait returned an error: context deadline exceeded"}
WARN deployment/chart.go:137 Chart installation failed. {"chart": "nsx-metadata-service-evaluation", "error": "client rate limiter Wait returned an error: context deadline exceeded"}
INFO tracker/tracker.go:64 Tracker timed out waiting for resources to be realized for chart : {"chartName": "nsx-metadata-service-evaluation", "StartTime": "2025-08-06T12:09:10.559Z", "CurrentTim
e": "2025-08-06T12:39:13.777Z", "OverallTimeout": 1800}
Error occurred while installing chart {"chartName": "nsx-metadata-service-evaluation", "error": {"Key":"error","Type":26,"Integer":0,"String":"","Interface":{}}}
ERROR install/install.go:59 Error occurred while installing chart. {"ChartName": "nsx-metadata-service-evaluation", "error": "Timed out waiting for resources to be realized for chart nsx-metadata-serv
ice-evaluation"}
ERROR install/install_service.go:22 Error occurred while installing: {"feature ": "ndr", "error": "Timed out waiting for resources to be realized for chart nsx-metadata-service-evaluation"}
ERROR cmd/installFeature.go:45 Failed to install feature {"Feature": "ndr", "error": "Timed out waiting for resources to be realized for chart nsx-metadata-service-evaluation"}
INFO feature/deployer.go:137 Feature installation failed {"Feature": "ndr"}
ERROR cmd/installFeature.go:45 Installation failed for feature {"Feature": "ndr", "error": "1 deployment (nsx-metadata-service) failed."}
Error: 1 deployment (nsx-metadata-service) failed.
Resolution
Before NDR can be re-installed, wait for the initial attempt to time out and fail (this may take up to an hour). After that, in the System -> Platform & Features view of the SSP UI, the NDR card will show that the deployment is in a failed state. Click DELETE on the card to uninstall NDR.
Before installing NDR again, follow the steps below to ensure the issue does not occur again:
- To install NDR with internet connectivity:
Ensure that internet connectivity is available (either directly or through a proxy). Additionally, ensure that the required domains are accessible from SSP, specifically *.prod.nsxti.vmware.com. If a proxy is required to access the internet, make sure it is configured in the SSP system settings before installing NDR. - To install NDR without internet connectivity in an Air-Gap environment:
NDR can operate in Air-Gap mode, where no internet connectivity is required. However, this mode must be explicitly selected by running a script. For instructions, refer to the user manual:
Configuring Network Detection and Response for the Air-Gap Environment
Feedback
