TEP IP Addresses Not Released After FORCE Deleting Host/Edge Transport Node in NSX-T UI

Products

• VMware NSX

Issue/Introduction

• When performing a FORCE delete operation on a transport node via the NSX-T UI, you may find that the Tunnel End Point (TEP) IP addresses are not released back into the IP pool. This can lead to issues with adding new hosts due to insufficient free IP addresses. This issue can also occur on NSX edge nodes.
• IP addresses remain allocated in the pool despite the FORCE delete operation.
• Unable to add a new Host/Edge due to exhausted IP pool.
• Checking the IP pools allocations after the force delete shows no IP addresses were released back into the pool:

GET /policy/api/v1/infra/ip-pools/<pool-uuid>/ip-allocations/

• If there are not enough free IP addresses in the pool, adding the host back again fails, as there is no free IP addresses in the pool, which can be seen in the NSX-T manager log /var/log/proton/nsxapi.log:

ERROR L2HostConfigTaskExecutor2 IPAMUtils 4433 FABRIC [nsx@6876 comp="nsx-manager" errorCode="MP8212" level="ERROR" subcomp="manager"] Failed to get a valid IP from IpPool IpPool/<IP-Pool-UUID> with cidr null, {}.

...

ERROR L2HostConfigTaskExecutor5 IPAMServiceImpl 4433 POOL-MGMT [nsx@6876 comp="nsx-manager" errorCode="MP5109" level="ERROR" subcomp="manager"] Insufficient Resource Exception : no free IP found to allocate from the pool IpPool/<IP-Pool-UUID>

Environment

VMware NSX-T Data Center 3.x
VMware NSX 4.x

Cause

The issue arises because a FORCE delete operation does not automatically release the TEP IP addresses back into the IP pool.

Resolution

This issue is resolved in:

• VMware NSX 4.1.1
• VMware NSX 4.2.0

Workaround:

To manually release the IP addresses from the TEP pool, follow these steps:

Method 1: Using API Calls from "Postman" or any API client

• Check IP Pools and get IP pool ID:

         GET https://<nsx-ip>/api/v1/pools/

• Verify the IPs being used in the pool

         GET https://<nsx-ip>/api/v1/pools/ip-pools/<IP-Pool-UUID>/allocations

• Release IP Addresses:

  Use the following API call to release specific IP addresses from the pool:

  POST https://<nsx-ip>/api/v1/pools/ip-pools/<pool-id>?action=RELEASE

  Request Body:

  { "allocation_id": "192.1XX.X.3" }

  Note: Replace 192.1XX.X.3 with the IP address you wish to release from the pool.

• Example Screenshot:

         

• Verify IP Release:

  To confirm that the IP addresses have been released, use this API call:

  GET https://<nsx-ip>/api/v1/pools/ip-pools/<IP-Pool-UUID>/allocations

Method 2: Using curl Commands

• Check IP Pools and get IP pool ID:

         curl -k -u admin -H "Content-Type: application/json" -X GET https://<nsx-ip>/api/v1/pools/

• Check IP Allocation List:

  Retrieve the list of allocated IPs in the specified pool with:

  cur -k -u admin -H "Content-Type: application/json" -X GET https://<mgr-ip>/api/v1/pools/ip-pools/<pool-id>/allocations > filename.txt

• Check Discovered Nodes:

  Retrieve the list of discovered nodes with:

  curl -k -u admin -H "Content-Type: application/json" -X GET https://<nsx-api>/policy/api/v1/fabric/discovered-nodes > filename.txt

• Release IP Addresses:

  Use this curl command to release a specific IP address from the pool:

  curl -k -u admin -H "Content-Type: application/json" -X POST https://<nsx-ip>/api/v1/pools/ip-pools/<pool-id>?action=RELEASE -d '{"allocation_id":"<stale-ip>"}'

  Note: Replace <stale-ip> with the IP address you wish to release from the pool.

• Verify IP Release:

  To verify that the IP addresses have been released, use the following command:

  cur -k -u admin -H "Content-Type: application/json" -X GET https://<mgr-ip>/api/v1/pools/ip-pools/<pool-id>/allocations

Additional Information

If you continue to experience issues or need further assistance, please open a case with Broadcom support.