NSX Global Manager: Switchover fails with SM63 version mismatch error during Federation upgrade

Products

VMware NSX

Issue/Introduction

In a VMware NSX Federation environment, a manual site switchover (failover) from the Active Global Manager (GM) to the Standby Global Manager fails after the Standby GM cluster has been upgraded to a newer version.

Symptoms:

Attempts to perform a switchover trigger a 500 Internal Server Error.
The UI or API returns the following error message: "error_code": 37000, "error_message": "Site manager error. [SM63] Can not perform switchover because active/standby GM have different versions.", "module_name": "node-services" (Error code 530039)
Communication between GM clusters appears restricted to read-only or synchronization-only operations.

Environment

VMware NSX
NSX Federation with Active and Standby Global Manager clusters

Cause

The NSX Site Manager service explicitly prevents manual site role switchovers while the Global Manager clusters are running different software versions.

During an upgrade, the Standby GM is upgraded first. While the Active GM remains on the older version, their configuration schemas and internal API versions do not match. Allowing a switchover to a newer version while the federation is in a transient state could lead to database inconsistencies or synchronization corruption. This protection mechanism is expected behavior during the Federation upgrade window.

Resolution

During an NSX Federation upgrade, a role switchover is not required and should not be performed. The upgrade of the Active GM cluster should be performed directly.

Do not attempt a site switchover.
Log in to the Active Global Manager UI.
Navigate to the System > Lifecycle Management > Upgrade tab.
Proceed with the upgrade of the Active Global Manager cluster directly.
Once both Active and Standby clusters are on the same version, switchover functionality will be restored.

Synchronization Check (If Active GM does not recognize Standby as Upgraded)

If the Active GM fails its own upgrade pre-checks because it incorrectly believes the Standby GM is not yet upgraded, force a configuration refresh:

Log in to any node of the Standby Global Manager cluster as the root user via SSH.

Run the following command to synchronize the site configuration:

curl -X POST -ik http://localhost:7999/api/v1/sites?action=refresh -H 'X-NSX-Username:admin;'

Return to the Active Global Manager UI and re-run the upgrade pre-checks.