IPSec VPN tunnels can go down abruptly with reason as "Peer not responding", when same IP address is used for DNAT and IPSec local endpoint.

Products

• VMware NSX

Issue/Introduction

Symptoms:

IPSec tunnel is down with reason as "Peer not responding".

edge04> get ipsecvpn session negotiating local-ip A.B.C.209 remote-ip F.G.H.134
Tue Jul 11 2023 UTC 21:08:09.399
Total Number of Negotiating Sessions: 1

IKE Session ID   : 8196
UUID             : ########-####-####-####-########e382
SR ID            : ########-####-####-####-########e567
Type             : Policy
Auth Mode        : PSK
Compliance Suite : NONE

Local IP         : A.B.C.209    Peer IP          : F.G.H.134
Local ID         : A.B.C.209    Peer ID          : F.G.H.134
Session Status   : Negotiating        Last Known Failure: Peer not responding

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

 

Environment

VMware NSX-T Data Center

Cause

When same IP address is used for DNAT and IPSec local endpoint and DNAT rule does not specify a service port at the destination IP, the IKE packets can be NATed leading to tunnel down.

Example: 

• Rule 536880363 is a DNAT for NTP, but the services was set to any for the destination IP, while Rule 536877071 below has the services port specified at the Destination IP.
• At the FW connections, we can see the IKE Packets getting NATed to the internal IP 10.2.2.2 on port 123

From the Edge bundle log: /edge

less /edge/fw-ruleset
"dnat": "
rule 536877071 at 1 in protocol tcp prenat from any to ip A.B.C.209 port 443 dnat ip 10.1.1.1 port 443; 
rule 536880363 at 2 in protocol udp postnat from any to ip A.B.C.209 dnat ip 10.2.2.2 port 123; ",

Take a closer look at the portion highlighted in green, we did not specify the NTP 123 port for the destination IP. As a result, IKE packets are getting NATed incorrectly.
 

less /edge/fw-connections
0x0c02fd7af4000001 af 2  ethertype 0x0000 proto udp   D.F.G.134:500 -> 10.2.2.2:123 (A.B.C.209:500) dir 1 2a1c 0 16 0 f-6374 n-536880363 flg:a00000040303 if:056a0be9a5c549ee:a5a1ff620b59e220 age:1:12543722

 

Resolution

This is Not a Bug but a configuration that disturbs IPSec tunnel. The remedy is to configure NAT properly.

Workaround:
When same IP address is to be used for NAT and IPSec, ensure that NAT rule is excludes IPSec Control traffic (UDP port 500/4500).
The workaround therefore is to configure NAT rule with relevant services only (not generic NAT rule).

Once corrected, dnat rule should look as below: 

rule 536880363 at 2 in protocol udp postnat from any to ip A.B.C.209 port 123 dnat ip 10.2.2.2 port 123

Additional Information

All NSX-T releases

Impact/Risks:
Disruption of traffic that is supposed to get encrypted with IPSec.