ICAP health check fails. Incomplete arp entry.

Products

• ISG Proxy
• ProxySG Software - SGOS

Issue/Introduction

Edge SWG (ProxySG) is generating alert ICAP service is not healthy. 

ICAP server is in the same subnet as proxy interface.

Health check statistics indicating:

  icap.icapservicename

    Enabled      Check failed      DOWN

  last status: Unable to connect to service.

Arp table indicates (cli command "show arp") incomplete mac address for the IP:

IP Address             MAC Address     Interface        Expiry
   X.X.X.X                (incomplete)          X:X            expired

 

Cause

ProxySG cannot learn the mac address of the ICAP server connected to the same subnet with the Proxy. 

Resolution

Collect the packet capture from the proxy to see what happens on the packet level:

- Set the Pcap filter on affected proxy and put the ICAP server IP as the host. Save settings and run the packet capture.

arp or ip host x.x.x.x

- Login to the CLI console via SSH, clear the ARP cache and run ping command:

clear-arp

ping 192.168.1.36

- Stop the packet capture and open it using Wireshark.

Search the ARP packets in the packet capture. Normally the ARP should indicate ARP requests from the proxy and responses from the ICAP servers:

X    XXXX-XX-XX XX:XX:XX.XXXXXXXXX    XX.XXXXXXXXX    XXXX_xx:xx:xx    Broadcast            ARP    XX                        Who has X.X.X.X? Tell Y.Y.Y.Y            
X    XXXX-XX-XX XX:XX:XX.XXXXXXXXX    XX.XXXXXXXXX    XXXXXXXXXX    XXXX_xx:xx:xx    ARP    XX                        X.X.X.X is at XX:XX:XX:XX:XX:XX            

If response is missing, engage network team to make sure ARP request can reach the ICAP server, as well as response can reach the proxy.

If different hosts are replying to the same ARP query, remove duplicated IP's from the subnet.

Additional Information

Use advanced URL's to get more details on failed service:

https://proxyip:8082/health_check/view
https://proxyip:8082/health_check/statistics