Using Google search returns "We're Sorry" page when going through a ProxySG

Products

ProxySG Software - SGOS

Issue/Introduction

Using Google search returns "We're Sorry" page when going through a ProxySG
Google sorry page is returned when going through a ProxySG
How do I fix my problem with the Google sorry page?

Resolution

The Google "We're Sorry" page occurs when automated/scripted searching, bot type behavior, malware or a worm, is detected on Google's servers. The ProxySG can appear to be the source of the problem because of network topology and deployment method used. But using Blue Coat's ProxySG by itself should not cause the Google sorry pages to appear. The key to successfully resolving the issue is to find the offending workstation(s), server(s), or network(s) and removing them from the network. Then the devices need to be cleaned of any malware installed on it.

TROUBLESHOOTING TOOLS AND TIPS

1.) Network topology: Transparent deployment / explicit deployment / NATing.

In a transparent deployment, all the intercepted traffic will flow through the proxy. When the ProxySG intercepts network traffic and sends it out to the Internet, the source IP address of the packet will be the ProxySG. If there are 1,000 workstations behind the proxy and one or two of the workstations are infected with a virus or worm, from Google's point of view, all traffic coming from the proxy's IP address appears compromised. Sometimes if the ProxySG is set to bypass the traffic (or reflect-client-ip is enabled), the Google sorry pages will cease to display for the end users. And if the ProxySG is set to intercept again, the Google sorry pages will return. This is expected behavior. Search for the upstream device or network that is sending the undesirable traffic, remove the device(s) from the network and the problem will clear itself up. If the problem does not go away if the proxy is removed from the equation, then look at your network topology. If the proxy is forwarding traffic upstream and is being NATed, some device on an unrelated network may be causing the problem.
If this is an explicit deployment, then all workstations and server will send their data to the proxy. This scenario is similar to the transparent deployment because the proxy will send out its own IP address as the source IP address. Again, the key is to find the offending devices.
NAT: This is where all the upstream network addresses are translated into one IP address. The proxy acts like a NAT because the proxy sends out client requests with the proxy's IP address. Or there may be upstream devices that will NAT the proxy's IP address. If the proxy is getting NATed upstream and there is no network traffic going to Google from the ProxySG, then the problem may be caused by an unrelated network or workstation.

2.) Use access logs and Blue Coat Reporter to determine what device or network is causing the problem.

3.) Review source IP addresses from access logs. If there are source IP addresses that are not from the network, such as external networks, look for an open proxy, a firewall problem, or a weak point into the network.

4.) Packet capture - This can potentially help find the offending workstation/server/network.

5.) Check internally and see if anyone is doing some testing or scripting which involves automated searches/scripts to Google. Items 2 and 4 should help identify if this is the problem.

6.) If you feel that the sorry pages are caused by the volume of traffic on your network and that there is nothing unusual is going on, you can contact Google at http://www.google.com/support/websearch/bin/request.py?contact_type=ban and see if they will lift the sorry page restriction.

OTHER RESOURCES:

Explanation of Google Sorry page from Google proper: http://googleonlinesecurity.blogspot.com/2007/07/reason-behind-were-sorry-message.html

Google Terms of Service: http://www.google.com/accounts/TOS