URL accessible with a "curl" test, and others, from CLI, but not accessible from any web browser

Products

• ISG Proxy

Issue/Introduction

When displayed on the CLI, the websites are accessible and they work. However, when accessed from just any browser, it reports that the page is not accessible (and the document size, from the developer tool, is 0).

Checks showed that the affected URL is public, and it is categorized as "Brokerage/Trading".

Environment

ISG-Proxy ver. 7.3.17.1

Cause

Investigation confirmed the following root causes, from implementation and from the end user environment setup:

1. Missing/unmatched policy, within the Proxy.
   
   The policy trace debug shoed very consistent "miss", for every occurrences of the reported URL. This article depicts the reported URL as <affected_URL>. See the policy trace debug excerpt below.
   
   miss:     client.connection.ssl_server_name=<affected_URL>
   
   miss:     url.domain=//<affected_URL>
   
   
2. With the missing policy rules eventually implemented, and tests done, it was further found that the user agent (browser) was not properly hardened, and was using TLS v1.0 by default, while the web server required TLS v1.2, and would block weaker connections.

Resolution

The following resolution steps were executed to resolve the issue, from within the end user's and from the Proxy:

1. Implemented requisite policy rule, in the web access layer, with the destination object speciifically referencing the identified URL Category, "Brokerage/Trading".
   
   
2. The user agent (browser) was properly hardened to support TLS v1.2 & TLS v1.3.
   
   
3. The following CPL script was aadded to the policy.
   
   <SSL>
    url.domain="<affected_URL>" server.connection.min_ssl_version(tlsv1.2) server.connection.max_ssl_version(tlsv1.3)

With the above steps executed accordingly, the affected URL became accessible.