Storage deletion on one NSX manager node results in NSX manager cluster becoming unresponsive

Products

• VMware NSX

Issue/Introduction

Symptoms:

If storage is deleted on one of the NSX managers, the NSX manager cluster becomes unreachable / unresponsive.

Cause

User deletion of storage / storage configuration

Resolution

This kind of failure (deleting Manager node Hard Disk in vCenter) is NOT something Corfu handles currently. See Workaround section for more information on other disk failure protection.

Workaround:

As of NSX 4.0, storage on one node can go into read-only mode. Corfu detects it is read-only mode then it removes that node from the cluster and continues to function.
In the event of user storage deletion, if entire cluster become unresponsive then we need to power-off the faulty node or disconnect its network to make the cluster run with the other two Manager nodes.

Additional Information

Impact/Risks:

The Manager cluster shows as Unresponsive, 'get cluster status' does not return output in the CLI, the NSX UI does not load.