Traffic failover when NSX Edge is placed in NSX Maintenance Mode (MM)
Article ID: 374750
Updated On:
Products
Issue/Introduction
When an Edge is placed in NSX MM, traffic failover (SR failover) happens with switching of Backplane IP.
Environment
VMware NSX
VMware NSX-T Data Center
Resolution
Lab validation:
Edge 01 :
Interface : 44####52-9e##-43##-83##-d6de####ddae Ifuid : 29# Name : bp-sr0-port Fwd-mode : IPV4_ONLY Internal name : backplane-29# Mode : lif Port-type : backplane IP/Mask : 169.###.#.2/25;fe##::##:56##:fe##:53##/64(NA) MAC : 02:50:##:##:##:00 VNI : 747## Access-VLAN : untagged LS port : 80####63-a4##-4a##-bd##-fe21####b434 Urpf-mode : NONE DAD-mode : LOOSE RA-mode : RA_INVALID Admin : up Op_state : up Enable-mcast : True MTU : 1500 arp_proxy :
Edge 02 :
Interface : b5####b8-18##-4d##-ad##-61aa####dc4a Ifuid : 27# Name : bp-sr1-port Fwd-mode : IPV4_ONLY Internal name : backplane-27# Mode : lif Port-type : backplane IP/Mask : 169.###.#.3/25;fe##::##:56##:fe##:53##/64(NA) MAC : 02:50:##:##:##:01 VNI : 747## Access-VLAN : untagged LS port : 36####1f-23##-4b##-a7##-08f8####ef6a Urpf-mode : NONE DAD-mode : LOOSE RA-mode : RA_INVALID Admin : up Op_state : up Enable-mcast : True MTU : 1500 arp_proxy :
The tenant ESXi host has both the Edges in its neighbor table pointing to its respective SR backplane interface mac-address.esxi> get logical-router 7a####55-43##-4c##-8b##-e906####11bc neighbor
Thu Aug 01 2024 UTC 11:43:12.474 Logical Routers Neighbors----------------------------------------------------------------------------------------------------------------------------------Flags Legend: [S: Static], [V: Valid], [P: Proxy], [I: Interface][N: Nascent], [L: Local], [D: Deleted], [K: linKlif], [A: AutoRefresh]
Network Mac Flags State Uptime SrcPort Refcnt Lif UUID========================================================================================================================================169.###.#.3 02:50:##:##:##:01 V 453 147 221####526 2 4b####aa-35##-44##-b8##-7f37####5b86169.###.#.2 02:50:##:##:##:00 V 536 2417 221####526 2 4b####aa-35##-44##-b8##-7f37####5b86
Tier-0 gateway in active-active mode showing two default routes in the forwarding table with gateway pointing to both edges.esxi> get logical-router 7a####55-43##-4c##-8b##-e906####11bc forwarding
Thu Aug 01 2024 UTC 11:43:41.136 Logical Routers Forwarding Table--------------------------------------------------------------------------------------------------------------Flags Legend: [U: Up], [G: Gateway], [C: Connected], [I: Interface][H: Host], [R: Reject], [B: Blackhole], [F: Soft Flush], [E: ECMP]
Network Gateway Type Interface UUID==============================================================================================================0.0.0.0/0 169.###.#.2 UGE 4b####aa-35##-44##-b8##-7f37####5b860.0.0.0/0 169.###.#.3 UGE 4b####aa-35##-44##-b8##-7f37####5b86
Edge 01 is now placed in NSX MM:
Edge 02 takes over Backplane Interface IP from Edge 01. The new IP-MAC association (169.###.#.2/25 & 02:50:##:##:##:01) will be broadcasted to the tenant ESXi host using GARP.
The Tier-0 gateway SR Backplane Interface sends 1 GARP Request and 10 GARP Replies. The GARP Replies are sent at an interval of 2 seconds. The request and replies, both are sent by the Edge Node which has resumed the Backplane Interface IP-address from the Edge Node that was placed in NSX Maintenance Mode (NSX MM).
Edge 02 :
Interface : b5####b8-18##-4d##-ad##-61aa####dc4a Ifuid : 27# Name : bp-sr1-port Fwd-mode : IPV4_ONLY Internal name : backplane-27# Mode : lif Port-type : backplane IP/Mask : 169.###.#.2/25;169.###.#.3/25;fe##::##:56##:fe##:53##/64(NA);fe##::##:56##:fe##:53##/64(NA) MAC : 02:50:##:##:##:01 VNI : 747## Access-VLAN : untagged LS port : 36####1f-23##-4b##-a7##-08f8####ef6a Urpf-mode : NONE DAD-mode : LOOSE RA-mode : RA_INVALID Admin : up Op_state : up Enable-mcast : True MTU : 1500 arp_proxy :
The tenant ESXi host updated its neighbor table using GARP broadcast.esxi> get logical-router 7a####55-43##-4c##-8b##-e906####11bc neighborThu Aug 01 2024 UTC 11:48:20.224 Logical Routers Neighbors----------------------------------------------------------------------------------------------------------------------------------Flags Legend: [S: Static], [V: Valid], [P: Proxy], [I: Interface][N: Nascent], [L: Local], [D: Deleted], [K: linKlif], [A: AutoRefresh]
Network Mac Flags State Uptime SrcPort Refcnt Lif UUID========================================================================================================================================169.###.#.3 02:50:##:##:##:01 V 507 455 221####526 2 4b####aa-35##-44##-b8##-7f37####5b86169.###.#.2 02:50:##:##:##:01 V 590 2725 221####526 2 4b####aa-35##-44##-b8##-7f37####5b86
Additional Information
For troubleshooting when the standby Edge Node IP does not appear, see KB 376948 – Troubleshooting NSX Edge High Availability
Feedback
