During NSX upgrade the Management Plane node failed to enter maintenance mode

Products

• VMware NSX

Issue/Introduction

• During NSX upgrade from version 4.2.0 to 4.2.1.3, the Management Plane node failed to enter maintenance mode
  
  
• In the /var/log/cloudnet/ccp.log on the NSX manager, you see an error similar as below
  2025-04-12T18:03:03.630Z INFO GmleClientBlockingOpsThread-6 MaintenanceModeHelper 226873 - >[nsx@6876 comp="nsx-manager" level="INFO" s2comp="maintenance-mode-helper" subcomp="ccp"] Updating maintenance mode status to TRANSITIONING_INTO_MAINTENANCE_MODE in GroupMaintenanceMode for memberId ########-####-####-####-###########, groupId ########-####-####-####-###########
  
  
• Run the below API to check the status of the maintenance mode
  GET https://NSX-MGR-IP/api/v1/cluster-manager/group-maintenance-mode
{
     "group_membership_update_ack_received": false,
     "leadership_work_completed": true,
     "maintenance_mode_status": "TRANSITIONING_INTO_MAINTENANCE_MODE",
     "member_id": "########-####-####-####-############"
}

Environment

VMware NSX 4.2.0

Cause

The Central Control Plane is stuck in the old version, and the sharding table voting process gets stuck.

 

Resolution

To resolve the issue, restart the "nsx-ccp" service by running the command below from the NSX manager node. 

/etc/init.d/nsx-ccp restart

Note: You should only run the service restart on the affected NSX manager node

Run the command below as an admin to check the node maintenance mode status. 

get group maintenance-mode status

Once the MP node has entered maintenance mode, retry the NSX upgrade.

If the issue persists or if the NSX manage node is unresponsive, you may refer to this KB to detach the NSX manager node and redeploy a new one then proceed with upgrade again. 

Additional Information

Note: It may take a minute for the node to go into maintenance mode after the CCP service restart.

In case you run into an issue with similar error "Failed to exit node <Manager UUID> from maintenance mode. Please retry the operation", please refer to below KB.
Unexpected error while upgrading upgrade unit. Failed to Exit node <manager uuid> from maintenance mode. Please retry the operation