Host disconnected with status unknown while all agents show up in NSX UI
search cancel

Host disconnected with status unknown while all agents show up in NSX UI

book

Article ID: 452815

calendar_today

Updated On:

Products

VMware NSX

Issue/Introduction

  • Host transport node shows NSX Configuration as Host disconnected and Status Unknown with Tunnels Not Available.
  • When checking the environment, no TEP tunnels are actually down, which can be verified by running the following command:
    nsxdp-cli bfd sessions list
  • vMotion of VMs might fail together with connecting VMs vNICs.
  • The transport node is connected to a controller:
    nsxcli -c get controllers
    Thu Aug 13 2026 UTC HH:MM:SS.MS
     Controller IP    Port     SSL         Status       Is Physical Master   Session State  Controller FQDN           Failure Reason
     ##.##.###.##    1235   enabled     connected             true               up               NA                       NA
     ##.##.###.##    1235   enabled      not used            false              null              NA                       NA
     ##.##.###.##    1235   enabled      not used            false              null              NA                       NA
  • The following errors and broken pipes are observed in the logs on the transport node in /var/run/log/nsx-syslog.log:
    YYYY-MM-DDTHH:MM:SS.MS+HH:MM Er(179) nsx-exporter[101267848]: NSX 101267848 - [nsx@6876 comp="nsx-esx" subcomp="agg-service" tid="101267909" level="ERROR" errorCode="MPA11006"] Error running script or empty output. rc=1, outLen=0, outBuffer=
    YYYY-MM-DDTHH:MM:SS.MS+HH:MM Er(179) nsx-exporter[101267848]: NSX 101267848 - [nsx@6876 comp="nsx-esx" subcomp="agg-service" tid="101267909" level="ERROR" errorCode="MPA11006"] HostCpuMemDataCollect() failed.
    
    YYYY-MM-DDTHH:MM:SS.MS+HH:MM Wa(28) cfgAgent[101267632]: YYYY-MM-DDTHH:MM:SS.MS+HH:MM|00106|jsonrpc|WARN|unix:/var/run/vmware/nsx-cfgagent/cfgagent-cli: send error: Broken pipe
    
    YYYY-MM-DDTHH:MM:SS.MS+HH:MM Wa(180) nsx-exporter[101267848]: NSX 101267848 - [nsx@6876 comp="nsx-esx" subcomp="nsx-fabric-exporter" s2comp="nsx-rpc" tid="101267912" level="WARNING"] RpcChannelScheduler[2] Latency observed in user dispatcher Processing time #### milliseconds
    
    YYYY-MM-DDTHH:MM:SS.MS+HH:MM Wa(180) cfgAgent[101267632]: NSX 101267632 - [nsx@6876 comp="nsx-controller" subcomp="cfgAgent" s2comp="nsx-core" tid="7DA10700" level="warn"] AsioNsxProvider: user dispatcher] 0x7036f956c8 blocked since #### seconds
    
    YYYY-MM-DDTHH:MM:SS.MS+HH:MM In(182) nsx-opsagent[101268106]: NSX 101268106 - [nsx@6876 comp="nsx-esx" subcomp="opsagent" s2comp="nsx-net" tid="101268113" level="INFO"] StreamConnection[1739 Closed to tcp://###.#.#.#:2480 sid:-1] Closed (reason: network error, error: 32-Broken pipe)
    YYYY-MM-DDTHH:MM:SS.MS+HH:MM Wa(180) nsx-opsagent[101268106]: NSX 101268106 - [nsx@6876 comp="nsx-esx" subcomp="opsagent" s2comp="nsx-rpc" tid="101268113" level="WARNING"] RpcConnection[1739 Negotiating to tcp://###.#.#.#:2480 0] WriteCallback - closing connection (error: 32-Broken pipe)
    YYYY-MM-DDTHH:MM:SS.MS+HH:MM Wa(180) nsx-opsagent[101268106]: NSX 101268106 - [nsx@6876 
  • nest-db communication channel is reported down:
    YYYY-MM-DDTHH:MM:SS.MS+HH:MM In(182) nsx-exporter[101267848]: NSX 101267848 - [nsx@6876 comp="nsx-esx" subcomp="nsx-fabric-exporter" s2comp="sndemux" tid="101267883" level="INFO"] Nestdb channel status turns to down
  • The NSX agents are up, but sub-agent cfgagent-nestdb-connection is reported to be down:
    YYYY-MM-DDTHH:MM:SS.MS+HH:MMZ In(182) nsx-sha: NSX 101268701 - [nsx@6876 comp="nsx-esx" subcomp="nsx-sha" username="root" level="INFO"]
    Dhreport - reporting, tn_agent_status:{"oid":{"left":"#########","right":"###########"},"status":[{"type":"TN_AGENT","tn_agent_status":{"threat_state":{},"agent_state":{"total_status":"UP"," ,"down_count":0,"agent_status":[{"type":"NSX_NESTDB","status":"UP","resource_stats":{"memory_used":"141384","memory_total":"2097152"}},{"type":"NSX_OPSAGENT","status":"UP","connection_status":[{"name":"opsagent-proxy-connection","status":"UP"}],"resource_stats":{"memory_used":"360684","memory_total":"1331200"}},{"type":"NSX_CFGAGENT","status":"UP","connection_status":[{"name":"cfgagent-nestdb-connection","status":"DOWN"}],"resource_stats":{"memory_used":"145872","memory_total":"6144000"}},{"type":"NSX_EXPORTER","status":"UP","resource_stats":{"memory_used":"171484","memory_total":"1536000"}},{"type":"NSX_VDPI","status":"UP","resource_stats":{"memory_used":"672084","memory_total":"1048576"}}],"degraded_count":0}}}]}

Environment

VMware NSX

Cause

The NSX agents on the host (nsx-cfgagent, nsx-exporter, and nsx-opsagent) are experiencing broken pipes, blocked dispatchers, and latency. This communication breakdown results in the host incorrectly appearing as disconnected.

If one or more agents are in "stopped" state, host transport nodes may come up as "Host Disconnected" in NSX UI.

Resolution

This is a condition that may occur in a VMware NSX environment.
To resolve this issue, perform the following steps:

  1. Restart the NSX agents on the affected ESXi host.
    /etc/init.d/<service name> status/stop/start

  2. If the issue persists after the agents have been restarted, restart the ESXi host itself.

Additional Information

Troubleshooting NSX Host Agents