How to update VMware NSX-T Edge hardware version using redeploy API

Products

VMware NSX

Issue/Introduction

An Edge Cluster may end up with different hardware versions in the following situations:

As result of upgrading ESXi/VC and then adding a new transport node to the cluster.
Two ESXi clusters hosting NSX-T Edges running different vSphere versions.

Environment

VMware NSX

Cause

NSX-T introduced hardware versions: vmx-17, vmx-19 and vmx-20 for edge virtual machines in NSX 4.0.1.
Edges which come as part of upgrade from any version prior to 4.0.1 have hardware version vmx-13.
From 4.1.0, Upgrade Coordinator supports update of hardware version for auto deployed Edge nodes during upgrade, only if the from version is 3.2.0 or higher and if deployed on ESXi host version 8.0. Hardware version of Edge virtual machine is updated to the max hardware version supported by the underlying VMware vSphere ESXi.

Resolution

Outside of an upgrade window, the hardware version of an auto-deployed Edge should be performed via redeploy of the Edge transport node.
It is recommended not to perform the hardware version upgrade from the vSphere client.
This operation impacts data traffic. A maintenance window to perform this operation is recommended.

From NSX 4.2.2.0 and above, auto deployed Edge VMs can be redeployed from the UI, System -> Fabric -> Edges. Actions, Redeploy -> Edge.

For earlier versions, Edge must be redeployed via API.

Determine which Edge needs to be redeployed

Pick an edge cluster against which hardware version mismatch alarm is raised. Using the following API, we can get the list of edge cluster members.

GET /api/v1/edge-clusters/<edge-cluster-id>

Check the current hardware versions of the edges in the edge cluster members using the following API:

GET /api/v1/transport-nodes/<transport-node-id>/state

{
   "transport_node_id":"<node-id>",
   "maintenance_mode_state":"DISABLED",
   "node_deployment_state":{
      "state":"NODE_READY",
      "details":[
          
      ],
      "failure_message":"",
      "failure_code"
   },
   "hardware_version":"vmx-19"
}

Using the above data, list the edges whose hardware version needs to be updated.
Update of hardware version can be done only via redeploy of the edge transport node. This operation will impact data traffic. A maintenance window to perform this operation is recommended.

Minimise disruption by identifying Edges running standby components

Using the following API , find what is the status of the Logical services and routers hosted on the edge transport node.

GET /api/v1/edge-clusters/<edge-cluster-id>/allocation-status

{
    "id": ",edge cluster id>",
    "display_name": "EDGECLUSTER1",
    "member_count": 2,
    "members": [
        {
            "member_index": 1,
            "node_id": "<node id>",
            "node_display_name": "nsxedge-ob-19232396-2-32",
            "allocation_pools": [
                {
                    "active_service_count": 0,
                    "standby_service_count": 0
                }
            ],
            "allocated_services": []
        },
        {
            "member_index": 0,
            "node_id": "<node id>",
            "node_display_name": "nsxedge-ob-19232396-1-32",
            "allocation_pools": [
                {
                    "active_service_count": 2,
                    "standby_service_count": 0
                }
            ],
            "allocated_services": [
                {
                    "service_reference": {
                        "target_id": ",target id>",
                        "target_display_name": "production_t1",
                        "target_type": "LogicalRouter"
                    },
                    "high_availability_status": "ACTIVE"
                },
                {
                    "service_reference": {
                        "target_id": ",<target id>",
                        "target_display_name": "it_t1",
                        "target_type": "LogicalRouter"
                    },
                    "high_availability_status": "ACTIVE"
                }
            ]
        }
    ]
}

Using the above data, choose edges with Tier1 standby logical routers first to redeploy to avoid an additional failover.

Redeploy the Edge

Redeploy the edge transport node:

- Get the current edge transport node configuration using

GET /api/v1/transport-nodes/<transport-node-id>

- Using the configuration obtained from the above api, redeploy the edge node.

POST /api/v1/transport-nodes/<transport-node-id>?action=redeploy

Post redeployment confirm the update of hardware version:

GET /api/v1/transport-nodes/<transport-node-id>/state

Use the allocation status API mentioned to make sure all the logical routers have an active instance up and running.

GET /api/v1/edge-clusters/<edge-cluster-id>/allocation-status

If all the logical routers hosted in this edge cluster are in serving state pick the next edge in the list populated.

Repeat the above workflow until all the edges in the edge cluster are in same hardware version.