Client.response.code:400 - Client Sends "Unknown" Request to ProxySG.

Products

• ProxySG Software - SGOS

Issue/Introduction

Your users are facing problems accessing the training site with 400 errors. Outside the network it is working fine.

Environment

SG/ASG/ISG-Proxy

Resolution

The uploaded policy trace and PCAP were looked at and we saw the below.

connection: service.name=Explicit HTTP client.address=xx.xx.xx.xx proxy.port=80 source.port=xxxxx dest.port=80 client.interface=2:0.1 routing-domain=default
  location-id=0 access_type=unknown
time: 2022-07-31 06:37:25 UTC
unknown <==   
user: unauthenticated
authentication status='none' authorization status='none'
client.host: <unset> (rdns resolution: lookup restricted by policy)
EXCEPTION(invalid_request): Request could not be handled
  Last Error: URL could not be parsed for matching purposes '' failed because: Scheme was not delimited by '://'
server.response.code: 0
client.response.code: 400
application.name: unavailable
application.operation: unavailable
application.group: unavailable
DSCP client outbound: 65
DSCP server outbound: 65

Transaction timing: total-transaction-time 53 ms
  Checkpoint timings:
    new-connection: start 1 elapsed 0 ms
    client-out-terminated: start 1 elapsed 0 ms
    access-logging: start 1 elapsed 52 ms
    stop-transaction: start 53 elapsed 0 ms
    Total Policy evaluation time: 52 ms
  url_categorization not completed
  client connection: first-response-byte 0 last-response-byte 1
stop transaction --------------------
start transaction -------------------
transaction ID=39662856 type=http.proxy

connection: service.name=Explicit HTTP client.address=xx.xx.xx.xx proxy.port=80 source.port=xxxxx dest.port=80 client.interface=2:0.1 routing-domain=default
  location-id=0 access_type=unknown
time: 2022-07-31 06:37:40 UTC
unknown <==
user: unauthenticated
authentication status='none' authorization status='none'
client.host: <unset> (rdns resolution: lookup restricted by policy)
EXCEPTION(invalid_request): Request could not be handled
  Last Error: URL could not be parsed for matching purposes '' failed because: Scheme was not delimited by '://'
server.response.code: 0
client.response.code: 400
application.name: unavailable
application.operation: unavailable
application.group: unavailable
DSCP client outbound: 65
DSCP server outbound: 65

Transaction timing: total-transaction-time 52 ms
  Checkpoint timings:
    new-connection: start 1 elapsed 0 ms
    client-out-terminated: start 1 elapsed 0 ms
    access-logging: start 1 elapsed 51 ms
    stop-transaction: start 52 elapsed 0 ms
    Total Policy evaluation time: 51 ms
  url_categorization not completed
  client connection: first-response-byte 0 last-response-byte 1
stop transaction --------------------
start transaction -------------------
transaction ID=39540727 type=http.proxy

While we do see a CONNECT request from the xx.xx.xx.xx client, as shown in the PCAP snippet below, the Proxy received only an "Unknown" request, as shown in the policy trace excerpt above.

Note 1: A "CONNECT" request urges the proxy to establish an HTTP tunnel to the remote end-point (xx.xx.xx.xx, in this case).

So, merely sending a CONNECT request isn't a guarantee that the Proxy receives the request. For a deployment that allows access to the Internet only via policy, seeing the CONNECT request in the Policy trace debug becomes the most potent proof that the ProxySG appliance actually receives the request and is able to process it. Clearly, from the two transactions that relate with the 10.50.xx.xx client, and with the reported "400" HTTP response code, we see that the ProxySG received an "Unknown' request. Following this, it is unable to process the request as either a http or an SSL request. Also, it does not receive any URL. It's just Unknown. In this case, the ProxySG would return the exception below, as seen in the trace.

EXCEPTION(invalid_request): The request could not be handled

Root Cause:  

Last Error: URL could not be parsed for matching purposes '' failed because: Scheme was not delimited by '://'

Ref. doc.: Troubleshooting the EXCEPTION(invalid_request) - failed

The above error simply means that the xx.xx.xx.xx client is able to send TCP packets to the Proxy but is sending incorrect data. To further understand the behavior of the packets, you may need to collect a PCAP directly from the xx.xx.xx.xx client. This is called the client-side PCAP. 

For the client PCAP, download and install Wireshark, (www.wireshark.org) on the client's PC prior to the test.  Click start capture and select the NIC that connects this workstation to the LAN, (and the proxy) when the policy trace, proxy packet capture and debug are in place.

Ref. doc.: 

Bluecoat packet captures for troubleshooting Edge SWG and Advanced Secure Gateway

Only to validate this issue, from the PCAP perspective (which would not still change the facts around the issue, as shared), we agreed, during the remote session, that you collect the PCAP again, this time with only the request for the reported/investigated issue sending packets to the Proxy, and share the same with us for validation. The requisite PCAP filter was already confirmed during the session. Usually, we would expect to see the "400 bad request or packet" in the network capture.

Note 2: The late tag in a policy trace means that the ProxySG appliance reached a verdict on the connection before it could evaluate the policy rules that otherwise would have been evaluated later in the connection. See Tech. Doc. with URL below. The late tag appeared multiple times in a policy trace because the ProxySG appliance terminated the transaction early in (before) the evaluation. In this case, all of the policy rules that were not evaluated are considered late. This went on, in cycles (loops), in the policy trace.

In a policy trace, what does the late tag mean

From the logs, we see, clearly, that the web requests failed not because of the web request but because of the invalid request, with invalid URL, sent by the client. Clearly, this is not an issue related to, or caused by, the ProxySG appliance.

We recommend that you investigate the affected source computers, in relation to the specific request. There may be settings/implementations of the particular source that's either unsupportive of a regular http requests or it's simply breaking it. You may test the same web request from another (clean) computer, in the same location.

Alternatively, if the challenge persists, the specific web request may be bypassed using a PAC file.