nsx-nestdb fails to start automatically during ESXi host boot/reboot

Products

• VMware NSX
• VMware NSX Networking

Issue/Introduction

• NSX was recently upgraded to 4.2.x.
• During ESXi host boot/reboot the nestdb agent remains down.
• nsxcli -c get controllers  returns  % Failed to get controller list
• NSX Manager > System > Nodes > [Node Name] > Monitor > Agent Status Down.
• Log lines similar to the below are encountered in /var/run/log/syslog.log on the ESXi host:

  In(14) jumpstart[2099923]: executing start plugin: nsx-pre-nestdb
  In(14) jumpstart[2099923]: executing start plugin: nsx-nestdb
  In(14) jumpstart[2099923]: nsx-nestdb started.
  In(30) NSX[2101624]: nsx-pre-nestdb started

  In sequence:
  1. Starting nsx-pre-nestdb.
  2. Starting nsx-nestdb (but nsx-pre-nestdb is not fully started yet).
  3. nsx-nestdb completes starting sequence (before nsx-pre-nestdb fully starts).
  4. nsx-pre-nestdb completes starting sequence. We are in the situation where nestdb did not start correctly.
• nsx-nestdb shows as stopped:
  

  /etc/init.d/nsx-nestdb status
  stopped

• There are no nsx-nestdb core dumps or other logging that indicates that nsx-nestdb has crashed.
• The Issue is intermittent. For some reboots the nsx-nestdb starts without issue.

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Environment

VMware NSX 4.2.x

Cause

esxcfg-info command in nsx-pre-nestdb script takes longer than expected to complete. This causes a delay in the execution of nsx-pre-nestdb which in turn affects nsx-nestdb as it completes startup before nsx-pre-nestdb.

Resolution

This issue is resolved in VMware NSX 4.2.1.1 available at Broadcom Downloads
Steps to locate and download Broadcom products and software are available at Download Broadcom products and software 

Workaround:

Manually start nestdb service with the following command:

/etc/init.d/nsx-nestdb start