NSX Host Upgrade Fails Config Agent (nsx-cfgagent) is Not Running on Host

Products

• VMware NSX

Issue/Introduction

• An NSX host upgrade fails for an ESXi host in a vSphere Lifecycle Manager (vLCM) enabled cluster. The following error is observed in the NSX UI:
  
  Upgrade failed: Upgrade not supported. Config Agent is not running on host. Run "/etc/init.d/nsx-cfgagent status" on the host to check the status and "/etc/init.d/nsx-cfgagent start" to start Config Agent. Upgrade failed: Solution apply failed on host: '<ESX_hostname>'


• CLI validation confirms the nsx-cfgagent service is not running. Attempts to manually start the service show it starting initially, but subsequent status checks reveal it stops immediately. The NSX UI may incorrectly report the nsx-cfgagent status as UP.
  
  [root@0ESX:~] /etc/init.d/nsx-cfgagent status
nsx-cfgagent service is not running
[root@0ESX:~] /etc/init.d/nsx-cfgagent start
nsx-cfgagent service starts
[root@0ESX:~] /etc/init.d/nsx-cfgagent status
nsx-cfgagent service is not running

  
• Check the /var/core directory to determine if the service has crashed or generated a core dump. In this scenario, no crash files will be present.
  
  
• Review the /var/run/log/nsx-syslog.log file for any application initialization failures. In this scenario, there will be no logs indicating an agent crash. Please refer Host Transport Node Shows as Down in NSX UI After Upgrade Due to Modified Configuration File.
  
  
• Validate the XML configuration by comparing /etc/vmware/nsx/nsx-cfgagent.xml from the affected host with a working host. Ensure there are no syntax errors, missing elements, or invalid XML tags. Please refer Hyperbus connection manager (nsx-cfgagent) service fails to start on ESXi host due to configuration parsing and After host upgrade from NSX 4.1.0.2 to 4.1.2.3, nsx-cfgAgent crashes.

Environment

VMware NSX 4.1.2.1 to 4.1.2.3
VMware vSphere ESXi
vSphere Lifecycle Manager (vLCM)

Resolution

To resolve this issue, follow the steps below:

• Place the affected ESXi host into Maintenance Mode.

• Reboot the ESXi host.

• Re-initiate the NSX upgrade for the host from the NSX UI once the host has fully booted and reconnected.

Additional Information

Upgrade Guide NSX 4.1 Upgrade guide provides step-by-step information about upgrading the VMware NSX components, which include the data plane, control plane, and management plane with minimum system downtime.