VMware NSX upgrade fails to place manager node in maintenance mode
search cancel

VMware NSX upgrade fails to place manager node in maintenance mode

book

Article ID: 322650

calendar_today

Updated On:

Products

VMware NSX VMware vSphere ESXi

Issue/Introduction

  • VMware NSX/VMware NSX-T Data Center is being upgraded from a VMware NSX 4.x version below 4.1.2
  • The upgrade target version is NSX 4.1.2 or higher.
  • During the Manager upgrade step, the upgrade fails if one of the NSX manager nodes cannot enter maintenance mode.
  • Clicking the VIEW link displays the error details stating: 
    "Unexpected error while upgrading upgrade unit: Failed to put node <name> in maintenance mode. Please retry the operation after checking 'get group maintenance-mode'..."
    
  •  "MAINTENANCE_MODE_FAILED" is seen on the NSX manager as admin when the following command is run:
    > get group maintenance-mode status
  • Results (below is a snippet of the result):
    Group Type: MANAGER
    Members:
        UUID                                  Leadership Work Completed   Group Update Ack Received   Maintenance Mode Status
        ########-65d4-4762-97af-############  False                       True                        MAINTENANCE_MODE_FAILED
        ########-c644-d0ab-6369-############  True                        True                        MAINTENANCE_MODE_OFF
        ########-2fdf-4fbd-bcc0-############  True                        True                        MAINTENANCE_MODE_OFF

Environment

VMware NSX-T Data Center 3.x
VMware NSX 4.x

Cause

As of VMware NSX 4.x, some services under the MANAGER (proton) group, such as VMC_* are no longer in use, however stale leases for those deprecated services (from older versions like 3.x) can still remain undeleted after upgrading to NSX 4.x. The presence of those stale leases prevent an NSX Manager appliance from entering into maintenance mode which causes an upgrade of the node to fail at that step. 

Resolution

This issue is resolved in VMware NSX 4.1.2 available at Broadcom Downloads.
Please refer the KB article  Download Broadcom products and software.

Note the fix applies to upgrades done from version 4.1.2 to a higher version.

If this issue is encountered and a workaround is required, please open a Broadcom Support request referencing this KB article