How Dynamic Categorization affects your private network

Products

• Advanced Secure Gateway Software - ASG
• ProxySG Software - SGOS

Issue/Introduction

You want to know what is sent to the WebPulse service when Dynamic Categorization is enabled, and if your private network is at risk? Or, what search results to expect when Dynamic Categorization is set to Real-time versus Background Mode.

Resolution

Dynamic Categorization processes requests in two modes:

• Real-time: A request to categorize the URL is sent to the WebPulse service and the ProxySG appliance waits for a response. The response, when found, is added to the rating cache and also used as the list of categories for the current request.
   
• Background: The ProxySG appliance continues to service the URL request without waiting for a response from WebPulse.

If a response is received, it is added to the rating cache, so future requests for that same URL will have the appropriate list of categories returned immediately. Reference to the site maybe recorded for future categorization in the BCWF database by automated background URL analysis or human analysis. (see Additional Information section for more details)

If a response is not received in a timely manner, or the request results cannot be categorized, nothing is added to the rating cache.

Depending on whether a category is found in BCWF or rating cache, you might see a variety of status results returned, such as None, Pending, Unlicensed, or Unavailable. The descriptions of these results is discussed in further detail in the SGOS 6.x Administration Guide.

But what is sent to the WebPulse service from your private network? Before a request is sent for content rating to the WebPulse cloud service, the following conditions are verified on the ProxySG appliance:

• Is WebPulse service and dynamic categorization enabled?
• Is dynamic categorization permitted by policy?
• Is the host specified in the private domain or private subnet list?
• Any request that is determined to be part of your configured private network is not sent to WebPulse.

When running a release older than SGOS 5.4.1, enabling WebPulse automatically sends certain customer information to the WebPulse service that cannot be modified.

With the introduction of SGOS 5.4.1 and later, private network domain names and IP subnets can be user-defined. Customer information sent to the WebPulse service is controlled by user-defined policy, although you can still use the default policy and configuration settings provided by the ProxySG appliance. Overriding the default settings with your organization’s policy definitions results in more control of the type of information that is sent to the WebPulse service. 

To learn more about Dynamic Categorization and how it impacts your private network, refer to the SGOS 6.x Administration Guide.