How to change the IP pool used by NSX Edge node TEPs

Products

• VMware NSX

Issue/Introduction

NSX Edge nodes are deployed using a specific IP pool for overlay networking TEP. This article provides steps to switch these nodes to a different IP pool when a subnet change or migration is required.

• NSX Edge nodes are currently deployed to use an IP pool.
• You want to switch the NSX Edge nodes to a different IP pool.

Environment

VMware VCF 9.x
VMware NSX 4.x
VMware NSX-T Data Center 3.x

Cause

A migration to a new IP pool is required due to network re-architecture or subnet exhaustion. Direct in-place changes may cause address conflicts or connectivity loss depending on subnet overlap and routing.

Resolution

We recommend performing the operation during a maintenance window, in accordance with your internal procedures, as dataplane impact is to be expected.
The configuration of an NSX Edge node cannot be edited while it is in Maintenance Mode.

Scenario 1: Subnet overlap

If the subnet of the new IP pool overlaps the subnet of the old IP pool, changing the IP pool in place may result in duplicate IP address allocated (one from the old IP pool, the same from the new IP pool).

1. Manually allocate all used IP addresses in the new IP pool via API call to reserve them.
2. Process each NSX Edge node individually:
   1. Unallocate the IP address used by the node in the new IP pool via API call.
   2. Edit the node configuration to select the new IP pool.
      Note: It may get a different IP address than initially depending on overlap. It is not an issue.

Scenario 2: No connectivity between subnets

If there's no connectivity between the subnets of both IP pools, changing individual nodes may result in impact as NSX Edge nodes will no longer be able to communicate on the overlay network (potentially affecting T0-T1 communication).

1. Deploy new NSX Edge nodes using the new IP pool.
2. Replace all standby NSX Edge nodes with new nodes first.
3. Then replace all active NSX Edge nodes with new nodes.

Scenario 3: Non-Overlapping Subnets with Connectivity

If there's no overlap between both subnets and they have network connectivity to each other, the impact would be minimal (convergence time only).

• • Edit the overlay switch on individual NSX Edge nodes in place.
  • Update the configuration to use the new IP pool.
  • Redeploy the NSX Edge node if the update does not take effect.

Additional information for individual steps

• Replacing an NSX Edge Transport Node in an NSX Edge Cluster
• Redeploy an NSX Edge Node
• Manually allocate an IP address in an IP pool
  PUT /policy/api/v1/infra/ip-pools/{ip-pool-id}/ip-allocations/{ip-allocation-id}
{
  "allocation_ip": "###.###.##.##"
}
• Manually unallocate an IP address from an IP pool
  DELETE /policy/api/v1/infra/ip-pools/{ip-pool-id}/ip-allocations/{ip-allocation-id}