Some or all VMware NSX Manager nodes fail to boot and display the following error message on the console screen:
"failed to start default target: transaction for nsx-custom.target/start is destructive (emergency.target has 'start' job queued, but 'stop' is included in transaction)"
In some scenarios. the NSX UI is inaccessible with error: Some appliance components are not functioning properly.
Component health: SEARCH:UNKNOWN, MANAGER:UNKNOWN, NODE_MGMT:UP, UI:UP.
Error code: 101
All services are running normally, and the Corfu Compactor Tool runs every 15 minutes without gaps.
Running the File System Check (fsck) tool does not resolve the issue.
VMware NSX
An environmental disruption, such as a storage/vSAN outage or a network outage, can cause corruption within the NSX Manager appliance file system or state.
A standard three-node NSX Manager cluster continues to be fully functional if a single node is lost. However, if two nodes are lost, the cluster loses its quorum (majority) and will no longer be operational. With only one remaining node unable to establish a majority, access to the UI is lost.
The official and supported procedure to recover an operational NSX node cluster is to deploy a new appliance, restore the environment from a valid backup, and then add two additional nodes to rebuild the cluster.
If a restore from backup is not immediately possible, use the following conditional workflows to attempt recovery depending on the state of the remaining manager nodes:
Access the NSX UI and navigate to System > Appliances.
Locate the problematic NSX Manager node and initiate the deletion process.
Once the problematic node is successfully removed, deploy a new node directly from the same interface page.
If only one NSX Manager node is showing as unavailable, follow the standard NSX product documentation to detach the nodes showing as unavailable.
If two NSX Managers or the entire cluster show as unavailable, follow the standard NSX product documentation to deactivate the NSX Manager cluster.
Check if the admin shell outputs the status of the surviving node as up or down (instead of unavailable) by running:
nsx-manager> get cluster status
Shut down the problematic node VMs.
Reboot the surviving NSX Manager node.
Run the get cluster status command on the surviving Manager node and wait for all services to transition to UP.
Once the NSX UI is confirmed to be operational, delete the broken manager node VMs from vCenter Server.
Verify that the NSX Manager cluster status displays as stable under System > Appliances.
Broadcom Support does not have a mechanism to restore an NSX environment if no valid backup is available.
If a restore from backup is impossible, open a support case referencing this KB article to have the situation assessed for potential alternative recovery options. For more information, see Creating and managing Broadcom support cases.