Traffic failover when NSX Edge is placed in NSX Maintenance Mode (MM)
Article ID: 374750
Updated On:
Products
Issue/Introduction
When an Edge is placed in NSX MM, traffic failover (SR failover) happens with switching of Backplane IP.
Environment
VMware NSX
VMware NSX-T Data Center
Resolution
Lab validation:
Edge 01 :
Interface : 44####52-9e##-43##-83##-d6de####ddae Ifuid : 292 Name : bp-sr0-port Fwd-mode : IPV4_ONLY Internal name : backplane-292 Mode : lif Port-type : backplane IP/Mask : 169.254.0.2/25;fe80::50:56ff:fe56:5300/64(NA) MAC : 02:50:56:56:53:00 VNI : 74753 Access-VLAN : untagged LS port : 80####63-a4##-4a##-bd##-fe21####b434 Urpf-mode : NONE DAD-mode : LOOSE RA-mode : RA_INVALID Admin : up Op_state : up Enable-mcast : True MTU : 1500 arp_proxy :
Edge 02 :
Interface : b5####b8-18##-4d##-ad##-61aa####dc4a Ifuid : 275 Name : bp-sr1-port Fwd-mode : IPV4_ONLY Internal name : backplane-275 Mode : lif Port-type : backplane IP/Mask : 169.254.0.3/25;fe80::50:56ff:fe56:5301/64(NA) MAC : 02:50:56:56:53:01 VNI : 74753 Access-VLAN : untagged LS port : 36####1f-23##-4b##-a7##-08f8####ef6a Urpf-mode : NONE DAD-mode : LOOSE RA-mode : RA_INVALID Admin : up Op_state : up Enable-mcast : True MTU : 1500 arp_proxy :
The tenant ESXi host has both the Edges in its neighbor table pointing to its respective SR backplane interface mac-address.esxi> get logical-router 7a####55-43##-4c##-8b##-e906####11bc neighbor
Thu Aug 01 2024 UTC 11:43:12.474 Logical Routers Neighbors----------------------------------------------------------------------------------------------------------------------------------Flags Legend: [S: Static], [V: Valid], [P: Proxy], [I: Interface][N: Nascent], [L: Local], [D: Deleted], [K: linKlif], [A: AutoRefresh]
Network Mac Flags State Uptime SrcPort Refcnt Lif UUID========================================================================================================================================169.254.0.3 02:50:56:56:53:01 V 453 147 2214592526 2 4b####aa-35##-44##-b8##-7f37####5b86169.254.0.2 02:50:56:56:53:00 V 536 2417 2214592526 2 4b####aa-35##-44##-b8##-7f37####5b86
Tier-0 gateway in active-active mode showing two default routes in the forwarding table with gateway pointing to both edges.esxi> get logical-router 7a####55-43##-4c##-8b##-e906####11bc forwarding
Thu Aug 01 2024 UTC 11:43:41.136 Logical Routers Forwarding Table--------------------------------------------------------------------------------------------------------------Flags Legend: [U: Up], [G: Gateway], [C: Connected], [I: Interface][H: Host], [R: Reject], [B: Blackhole], [F: Soft Flush], [E: ECMP]
Network Gateway Type Interface UUID==============================================================================================================0.0.0.0/0 169.254.0.2 UGE 4b####aa-35##-44##-b8##-7f37####5b860.0.0.0/0 169.254.0.3 UGE 4b####aa-35##-44##-b8##-7f37####5b86
Edge 01 is now placed in NSX MM:
Edge 02 takes over Backplane Interface IP from Edge 01. The new IP-MAC association (169.254.0.2/25 & 02:50:56:56:53:01) will be broadcasted to the tenant ESXi host using GARP.
The Tier-0 gateway SR Backplane Interface sends 1 GARP Request and 10 GARP Replies. The GARP Replies are sent at an interval of 2 seconds. The request and replies, both are sent by the Edge Node which has resumed the Backplane Interface IP-address from the Edge Node that was placed in NSX Maintenance Mode (NSX MM).
Edge 02 :
Interface : b5####b8-18##-4d##-ad##-61aa####dc4a Ifuid : 275 Name : bp-sr1-port Fwd-mode : IPV4_ONLY Internal name : backplane-275 Mode : lif Port-type : backplane IP/Mask : 169.254.0.2/25;169.254.0.3/25;fe80::50:56ff:fe56:5300/64(NA);fe80::50:56ff:fe56:5301/64(NA) MAC : 02:50:56:56:53:01 VNI : 74753 Access-VLAN : untagged LS port : 36####1f-23##-4b##-a7##-08f8####ef6a Urpf-mode : NONE DAD-mode : LOOSE RA-mode : RA_INVALID Admin : up Op_state : up Enable-mcast : True MTU : 1500 arp_proxy :
The tenant ESXi host updated its neighbor table using GARP broadcast.esxi> get logical-router 7a####55-43##-4c##-8b##-e906####11bc neighborThu Aug 01 2024 UTC 11:48:20.224 Logical Routers Neighbors----------------------------------------------------------------------------------------------------------------------------------Flags Legend: [S: Static], [V: Valid], [P: Proxy], [I: Interface][N: Nascent], [L: Local], [D: Deleted], [K: linKlif], [A: AutoRefresh]
Network Mac Flags State Uptime SrcPort Refcnt Lif UUID========================================================================================================================================169.254.0.3 02:50:56:56:53:01 V 507 455 2214592526 2 4b####aa-35##-44##-b8##-7f37####5b86169.254.0.2 02:50:56:56:53:01 V 590 2725 2214592526 2 4b####aa-35##-44##-b8##-7f37####5b86
Feedback
