Unsupported scenario in NSX-T IPSec VPN (Policy Based VPN) when multiple local and remote subnets are configured.

Products

VMware NSX

Issue/Introduction

Introduction:

When IKEv2 is used for negotiation of IPSec SA's between VPN endpoints, the protocol provides multiple ways to exchange local and remote subnets configured for the IPSec tunnel. The protocol defines Traffic Selector (TSi/TSr) payload to be used to specify local and remote subnets configured onboard each side. In one of the negotiation methods, the protocol allows multiple local and remote subnets to be specified in the Traffic Selector payload as part of a single exchange. This mechanism is not supported in NSX-T.

Condition:

There are multiple local and remote networks configured in a Policy based VPN session.
Peer device is not NSX-T Edge.

Symptoms:

Only one tunnel is UP and others are Down with reason as "IPsec negotiation not started".
Session status is seen as Degraded.
Initial IKE negotiations show only IKE AUTH message exchanges, no CREATE-CHILD-SA exchange is seen.

Vendor details:

So far the behaviour has been observed with Meraki VPN.

Environment

VMware NSX

Cause

When multiple local and remote subnets are configured (e.g., "N" local and "M" remote networks), NSX-T VPN attempts to create N * M internal tunnels. For each unique pair of local and remote subnets, a dedicated tunnel is generated, meaning each individual Traffic Selector (TS) request contains only one local and one remote network.
- Ex. N = 2 (Local networks: 192.#.#.0/24, 192.#.#.0/24) and M = 1 (Remote Networks: 172.##.##.0/24)
- In this case there will be 2 requests as part of two separate exchanges, one with TS 192.##.#.0/24 - 172.##.#.0/24 and second with TS 192.##.#.0/24 - 172.##.#.0/24.
However, a conflict arises if the Peer initiates a request and specifies all local and remote subnets within a single exchange, therefore trying to setup single IPSec tunnel for all the subnets. In this scenario, the Peer sends a single Traffic Selector Payload request containing all local and remote subnets. In such cases NSX-T shall setup tunnel only for the first combination appearing in the list as provided by Peer VPN endpoint.
Consequently, only one tunnel is brought UP at the NSX-T side instead of the expected N * M tunnels. Traffic only flows for that specific subnet pair, while all other configured tunnels remain in a 'Down' state because the Peer never sends individual requests for the remaining subnet combinations.

Resolution

Workaround:

Peer is configured as Responder.
Use IKEv1 if peer IKEv1 implementation is as per RFC and doesn't not support negotiating multiple selectors in single exchange.
If peer device has customised IKEv1 with such support, then workaround needs to be checked from peer device side.

Additional Information

More Information:

Behaviour of Edge in different scenarios:

Following are different scenarios with ex. packet exchanges for N=2, M=2.

Peer is initiator and Edge as Respond Only.

1. In this case, from the packet exchanges it can be seen that even if multiple local and remote networks are configured, only one request is received from Peer as part of IKE AUTH. There will be no separate CHILD_SA requests seen.
2. This results in Edge taking first matching local and remote subnet pair, and that particular tunnel is UP.

IKEv2 packet R(192.##.##.100:500 <- 20.##.##.101:500): len= 456, mID=0, HDR(SPI_i, SPI_r), SA, KE, Nonce, N(NAT_DETECTION_SOURCE_IP), N(NAT_DETECTION_DESTINATION_IP), N(FRAGMENTATION_SUPPORTED), unknown

IKEv2 packet S(192.##.##.100:500 -> 20.##.##.101:500): len= 462, mID=0, HDR(SPI_i, SPI_r), SA, KE, Nonce, N(NAT_DETECTION_SOURCE_IP), N(NAT_DETECTION_DESTINATION_IP), N(FRAGMENTATION_SUPPORTED), Vid

IKEv2 packet R(192.##.##.100:4500 <- 20.##.##.101:4500): len= 312, mID=1, HDR(SPI_i, SPI_r), IDi, N(INITIAL_CONTACT), IDr, AUTH, SA, TSi, TSr, N(EAP_ONLY_AUTHENTICATION)

IKEv2 packet S(192.##.##.100:4500 -> 20.##.##.101:4500): len= 228, mID=1, HDR(SPI_i, SPI_r), IDr, AUTH, SA, TSi, TSr, N(ESP_TFC_PADDING_NOT_SUPPORTED)

Peer is Respond Only.

1. In this case, Edge initiates N*M CHILD_SA requests which results in all tunnels UP.
2. But later if Peer initiates IPSec rekey, it can result in issue as case 1. Therefore in such cases rekey initiation from Peer side should be avoided. One way to achieve the same is by configuring a lower SA timer at the Edge side so that the Edge rekeys earlier than Peer.

IKEv2 packet S(192.##.##.100:500 -> 20.##.##.101:500): len= 462, mID=0, HDR(SPI_i, 0000000000000000_r), SA, KE, Nonce, N(NAT_DETECTION_SOURCE_IP), N(NAT_DETECTION_DESTINATION_IP), N(FRAGMENTATION_SUPPORTED), Vid

IKEv2 packet R(192.##.##.100:500 <- 20.##.##.101:500): len= 448, mID=0, HDR(SPI_i, SPI_r), SA, KE, Nonce, N(NAT_DETECTION_SOURCE_IP), N(NAT_DETECTION_DESTINATION_IP), N(FRAGMENTATION_SUPPORTED), N(MULTIPLE_AUTH_SUPPORTED)

IKEv2 packet S(192.##.##.100:500 -> 20.##.##.101:500): len= 416, mID=1, HDR(SPI_i, SPI_r), IDi, IDr, AUTH, SA, TSi, TSr, N(INITIAL_CONTACT), N(ESP_TFC_PADDING_NOT_SUPPORTED)

IKEv2 packet R(192.##.##.100:500 <- 20.##.##.101:500): len= 192, mID=1, HDR(SPI_i, SPI_r), IDr, AUTH, SA, TSi, TSr

IKEv2 packet S(192.##.##.100:500 -> 20.##.##.101:500): len= 656, mID=2, HDR(SPI_i, SPI_r), SA, Nonce, KE, TSi, TSr, N(ESP_TFC_PADDING_NOT_SUPPORTED)

IKEv2 packet R(192.##.##.100:500 <- 20.##.##.101:500): len= 448, mID=2, HDR(SPI_i, SPI_r), SA, Nonce, KE, TSi, TSr

IKEv2 packet S(192.##.##.100:500 -> 20.##.##.101:500): len= 656, mID=3, HDR(SPI_i, SPI_r), SA, Nonce, KE, TSi, TSr, N(ESP_TFC_PADDING_NOT_SUPPORTED)

IKEv2 packet R(192.##.##.100:500 <- 20.##.##.101:500): len= 448, mID=3, HDR(SPI_i, SPI_r), SA, Nonce, KE, TSi, TSr

IKEv2 packet S(192.##.##.100:500 -> 20.##.##.101:500): len= 656, mID=4, HDR(SPI_i, SPI_r), SA, Nonce, KE, TSi, TSr, N(ESP_TFC_PADDING_NOT_SUPPORTED)

IKEv2 packet R(192.##.##.100:500 <- 20.##.##.101:500): len= 448, mID=4, HDR(SPI_i, SPI_r), SA, Nonce, KE, TSi, TSr

Both Peer and Edge are Initiator.

1. In this scenario, results vary depending on whether the initiation is successful for one or both devices.

IKEv2 packet S(192.##.##.100:500 -> 20.##.##.101:500): len= 462, mID=0, HDR(SPI_i, 0000000000000000_r), SA, KE, Nonce, N(NAT_DETECTION_SOURCE_IP), N(NAT_DETECTION_DESTINATION_IP), N(FRAGMENTATION_SUPPORTED), Vid

IKEv2 packet S(192.##.##.100:500 -> 20.##.##.101:500): mID=0, (SPI_i, 0000000000000000_r)(retransmit count=1)

IKEv2 packet S(192.##.##.100:500 -> 20.##.##.101:500): mID=0, (SPI_i, 0000000000000000_r)(retransmit count=2)