If you observe "ICAP Queued Connections" on Edge SWG (ProxySG) where queued connections were observed in Data Loss Prevention (DLP) (Request Mode) or Content Analysis (Response Mode), where increase in queued connections to large number impacts Edge SWG (ProxySG) to becomes unresponsive with not accepting further new connection request from clients.
Microsoft Azure support DLP.
Proxy performing ICAP request scanning using DLP or Proxy performing ICAP response scanning using CAS/AV.
DLP Network Prevent for Web, aka "Web Prevent" / Content Analysis
It has been observed that if incoming request is something "infinite stream" (i.e. client uploading streaming data to OCS), those requests gets stuck between Proxy and DLP, where those requests could cause "Queued" connections on Proxy. This is expected behavior as Proxy and DLP/CAS device can not determine end of the data stream.
For Example, we have recently observed that, Request to/from "Microsoft Azure" (one example given below) causes the request to get stuck in "transferring state" and causes high amount of queued connections on proxy:
Above URL looks simple, but it has embedded "webstream" in it.
To avoid Queued Connections on Proxy, it is recommended to not send requests with "infinite stream" to the DLP/CAS/AV [Where request of this type does not know where to Terminate (infinite stream)] and causes Loop between Proxy & DLP.
All ICAP Connections configured on ProxySG gets used up and no more further connections are available to serve new requests.
If you are sending "HTTP/Main" logs from proxy to Reporter, then you can generate a report on Reporter base on Custom Date, Time and Method [PUT, POST].
Look for reports based on "Request" Field and Check for the "Highest Requests", based on that you can determine what "website" or "ServerIP" were requested the most (during queued connections ). Based on type of request if it is something "webstream" , you can make decision and should be able to determine the request causing problems.
If you notice end users are complaining with slowness or degradation in performance while accessing websites:
Statistics ---> Sessions --> Active Sessions ---> Proxies Sessions
Filter : ICAP, REQMOD and Click on Show
Search based on "Duration", if you notice any requests which are stuck in "Transferring" or "Scanning" when you hover mouse to "I" (internet content adoptation protocol). Those are the requests , which are causing queued connections and for some reason are not able to free up. [Due to type of request something like infinite streaming or infinite connection]
To Avoid Queued Connections caused due to Below Request which is related to Microsoft Azure, as we know its issue with request, kindly add below CPL code on ProxySG to avoid queueing:
url.host.is_numeric=yes , is because the IP Address is full CIDR , which keeps on changing, so we are not sending this request for scanning based on specific path in URL.
Attached is the latest ICAP best practice that include implementing REQMOD, DLP best practice.