Troubleshooting IP conflict with VRRP

Products

• XOS

Issue/Introduction

VRRP - Address duplication causing traffic failureWhen VRRP IP and another duplicate IP conflict on the network, traffic would stop getting forwarded on the box every 3 hrs.

However if VRRP were failed over then the problem was resolved.

Cause

The cause of the issue is the duplicate IP on the network. A previously assigned address was now being used by the recently deployed VRRP virtual router on the X chassis DBHA pair.

The reason that this was difficult to isolate was that VRRP is designed to duplicate IP's on multiple machine. If another machine ARPs (RARP's more specifically... replying to an ARP request), this is not logged by VRRPd in syslog.

When the customer experienced the problem it meant that the duplicate mac-address was in the router's ARP table and traffic was no longer routed to the desired firewall. The reason that the failover process resolved the issue is that during a VRRP failover a gratuitous ARP is issued. This forced an update of the ARP tables of the routers and resolved the problem.

Resolution

To force a gratuitous arp on an interface you can use the following command:

# arping -q -c 3 -A -I eth11 172.16.1.7
 

We were also able to isolate the issue using tcpdump and looking specifically for ARP requests:

# tcpdump -nni eth11 arp
 

These commands allow for sending and monitoring gratuitous ARP's out for an address in Linux which is extremely helpful when trouble shooting layer 2 issues.

Workaround

N/A