Hyperbus connection manager (nsx-cfgagent) service fails to start on ESXi host due to configuration parsing

Products

• VMware NSX

Issue/Introduction

• The Hyperbus connection manager service fails to start on an ESXi host. The /etc/init.d/nsx-cfgagent service continuously crashes and goes down immediately after any start or restart attempts.
• The NSX manager GUI may show the nsx_cfgagent service status as down, and users might receive alerts prompting a service restart.
• Checking the system logs (such as nsx-syslog.log) during service startup will typically indicate that the system is unable to access or parse the configuration file.

Environment

VMware NSX
VMware ESX

Cause

• This issue is usually caused by a syntax error introduced during a manual or automated modification of the nsx-cfgagent.xml configuration file on the ESXi host.
• Invalid XML tags prevent the ESXi service manager from successfully parsing the manifest, which causes the service initialization to fail and the service to crash.
  • For example, a typographical error in the XML structure, such as writing <enabled>falseled> instead of a valid closing tag, will trigger this failure.

Resolution

To resolve this issue, identify and correct the invalid XML formatting in the configuration file:

1. SSH into the affected ESXi host.

2. Open the nsx-cfgagent.xml file using a text editor (e.g., vi).

3. Review the file for any recent manual changes, malformed tags, or syntax errors. For example, look for invalid tags like the <enabled>falseled> typo mentioned above.

4. Correct the syntax error so that all XML tags are valid and properly closed (e.g., correcting the example typo to <enabled>false</enabled>).

5. Save the changes and exit the text editor.

6. Restart the nsx-cfgagent service using the following command: /etc/init.d/nsx-cfgagent restart

7. Verify that the service comes up and remains in a running state.

Note: If manual modifications or scripts were used to push changes across multiple hosts, it is highly recommended to audit other ESXi hosts in the environment for similar syntax errors. Even if the service is currently running on those hosts, it may be in a non-deterministic state and could fail upon the next restart.

Additional Information

Reference Link:
After host upgrade from NSX 4.1.0.2 to 4.1.2.3, nsx-cfgAgent crashes