How to update VMware NSX-T Edge hardware version using redeploy API

Products

• VMware NSX

Issue/Introduction

An Edge Cluster may end up with different hardware versions in the following situations:

• As result of upgrading ESXi/VC and then adding a new transport node to the cluster.
• Two ESXi clusters hosting NSX-T Edges running different vSphere versions. 

Environment

VMware NSX

Cause

• NSX-T introduced hardware versions: vmx-17, vmx-19 and vmx-20 for edge virtual machines in NSX 4.0.1.
• Edges which come as part of upgrade from any version prior to 4.0.1 have hardware version vmx-13.

From 4.1.0, Upgrade Coordinator supports update of hardware version for auto deployed Edge nodes during upgrade, only if the from version is 3.2.0 or higher and if deployed on ESXi host version 8.0. Hardware version of Edge virtual machine is updated to the max hardware version supported by the underlying VMware vSphere ESXi.

Resolution

Workaround:

Update of hardware version can be done only via redeploy of the edge transport node. This operation impacts data traffic. A maintenance window to perform this operation is recommended.

• Pick an edge cluster against which hardware version mismatch alarm is raised. Using the following API, we can get the list of edge cluster members.

GET /api/v1/edge-clusters/<edge-cluster-id>

• Find the hardware versions of the edges in the edge cluster members using the following API:

GET /api/v1/transport-nodes/<transport-node-id>/state

{
   "transport_node_id":"<node-id>",
   "maintenance_mode_state":"DISABLED",
   "node_deployment_state":{
      "state":"NODE_READY",
      "details":[
          
      ],
      "failure_message":"",
      "failure_code"
   },
   "hardware_version":"vmx-19"
}

• Using the above data, list the edges whose hardware version needs to be updated.
• Update of hardware version can be done only via redeploy of the edge transport node. This operation will impact data traffic. A maintenance window to perform this operation is recommended.
• Using the following API , we can find what is the status of the Logical services and routers hosted on the edge transport node.

GET /api/v1/edge-clusters/<edge-cluster-id>/allocation-status

{
    "id": ",edge cluster id>",
    "display_name": "EDGECLUSTER1",
    "member_count": 2,
    "members": [
        {
            "member_index": 1,
            "node_id": "<node id>",
            "node_display_name": "nsxedge-ob-19232396-2-32",
            "allocation_pools": [
                {
                    "active_service_count": 0,
                    "standby_service_count": 0
                }
            ],
            "allocated_services": []
        },
        {
            "member_index": 0,
            "node_id": "<node id>",
            "node_display_name": "nsxedge-ob-19232396-1-32",
            "allocation_pools": [
                {
                    "active_service_count": 2,
                    "standby_service_count": 0
                }
            ],
            "allocated_services": [
                {
                    "service_reference": {
                        "target_id": ",target id>",
                        "target_display_name": "production_t1",
                        "target_type": "LogicalRouter"
                    },
                    "high_availability_status": "ACTIVE"
                },
                {
                    "service_reference": {
                        "target_id": ",<target id>",
                        "target_display_name": "it_t1",
                        "target_type": "LogicalRouter"
                    },
                    "high_availability_status": "ACTIVE"
                }
            ]
        }
    ]
}

• Using the above data, customer can choose edges with Tier1 standby logical routers first to redeploy to avoid an additional failover.
• Redeploy the edge transport node:

• • Get the current edge transport node configuration using

GET /api/v1/transport-nodes/<transport-node-id>

• • Using the configuration obtained from the above api, redeploy the edge node.

POST /api/v1/transport-nodes/<transport-node-id>?action=redeploy

• Use API in Step 2 to confirm the update of hardware version:

GET /api/v1/transport-nodes/<transport-node-id>/state

• Use the allocation status API mentioned in step 5 to make sure all the logical routers have an active instance up and running.

GET /api/v1/edge-clusters/<edge-cluster-id>/allocation-status 

• If all the logical routers hosted in this edge cluster are in serving state pick the next edge in the list populated at step 3.

 

Repeat the above workflow until all the edges in the edge cluster are in same hardware version.

Additional Information

Impact/Risks:

Deploying different versions of NSX-T Edge virtual machines may result in performance differences depending on which Edge transport node is the active instance of the logical services hosted on the Edge