NSX upgrade failed with error: "Failed to put node <manager UUID> in maintenance mode" when upgrading from 4.2.1 to 4.2.3.1

Products

• VMware NSX

Issue/Introduction

This article provides a resolution for an issue where an NSX Manager upgrade fails because the node cannot enter into maintenance mode when upgrading from 4.2.x to 4.2.3.1. 

Symptoms

• NSX Manager upgrade fails at Node OS Upgrade section with Error message: "Unexpected error while upgrading upgrade unit. Failed to put node in maintenance mode." 

       

• Running get group maintenance-mode status shows Group Type: MONITORING with a status of MAINTENANCE_MODE_FAILED and  Group Update Ack Received  is at False

   

Environment

• VMware NSX 4.2.x
• Upgrading from 4.2.1.0 to 4.2.3.1

 

Cause

The phonehome-coordinator service (responsible for the Monitoring group) hangs or fails to initialize correctly, preventing the upgrade coordinator from transitioning the node's maintenance state.

Resolution

To resolve this issue, restart the monitoring service on the affected NSX Manager node by executing the following procedures:

1. Log in to the affected NSX Manager appliance as root via SSH.

2. Restart the phonehome-coordinator service:

   /etc/init.d/phonehome-coordinator restart
   

3. Verify the service is running:

   /etc/init.d/phonehome-coordinator status
   

4. Check the maintenance mode status from the admin CLI:

   get group maintenance-mode status
   

5. Retry the upgrade from the NSX Manager UI and the upgrade should proceed further

Additional Information

Reference Articles:

NSX Manager node upgrade failed at step "Entering maintenance mode".

VMware NSX upgrade fails to place manager node in maintenance mode