Common website categorization issues affecting Edge SWG
Article ID: 170495
Updated On:
Products
Issue/Introduction
The following will discuss some common website categorization issues, give you an explanation of each and point you to documentation containing further explanation of these issues.
Resolution
Table of Contents
Websites blocked by categorization policy
There are many issues that can be faced with site categorization and unintentional blocking of a site due to a categorization.
If a site is being unintentionally blocked due to its categorization you can start the process of troubleshooting this issue by following the instructions in the following documentation:
Troubleshooting unintentional URL blocking by content filtering on the Proxy SG
Sites can also be categorized into multiple categories. If your policy is allowing one of the categories but blocking the second category within the same web access layer in a later rule, the block rule will never be a match. The policy would have to be adjusted to take both categories of the website into account. See the following document for more details on this:
With Content Filtering, why do some URLs have more than one category?
There are situations where you will find that a subdomain of a website will have a different categorization than that of the main website.
for example:
https://www.yahoo.com/ is categorized as Search Engines/Portals
https://finance.yahoo.com is categorized as Financial Services and News/Media
Just as previously mentioned about sites having multiple categories. These types of sites can cause unexpected issues when creating policy.
Please see the following document to learn more:
Blue Coat Web Filter Service Sub-Domain Classification
Incorrect website categorization
When websites are categorized, as Malicious Outbound Data/Botnets, Malicious Sources/Malnets, Phishing, Potentially Unwanted Software, Scam/Questionable/Illegal, Spam, and Suspicious, there is usually a reason for this categorization. Third-party tools can be used to investigate the categorization of problem websites. These tools will reveal such things as invalid DNS entries for websites, if the website links to other websites containing malware, if the website is running a vulnerable version of the software, or if the website has malformed java which may be an indication of an infection. Once this information is known, then you can make an informed decision about whether you would like to submit a request to have the website recategorized and also if you would like to adjust the policy to allow the website. If you would like to check the categorization of a website and submit a website for recategorization then you can do so at WebPulse Site Review Request. The following documents will provide you with more information about submitting a website for recategorization and how the process works:
WebPulse Site Review Request Process
The site review website is only good if you are checking the category of one website URL. If you have a long list of URLs and you would like to find out what category these URLs belong to, then you can follow the instructions in the following document and you will have the information that you need in short order:
Alternate Ways to Identify Blue Coat Web Filter Categories for a long list of URLs
Differing results from dynamic real-time categorization results
The web is an ever-changing place and sites are being recategorized on a regular basis. The categories associated with the most popular websites remain somewhat static but there are some websites that can be recategorized depending on the type of website, the amount of traffic, and the type of content that it contains. There are times when a website may not appear in the on-box website categorization database. The site can still be categorized however by dynamic categorization via WebPulse. This dynamic categorization process can present its own set of challenges. Please review the following documentation to learn more about dynamic categorization and some of the issues associated with it:
Why do sitereview.bluecoat.com and the Proxy SG appliance sometimes return different categories for websites?
How Does Blue Coat WebPulse work with Blue Coat Web Filter?
About Dynamic Categorization and How It Impacts Your Private Network
Categorization subscription expiry
One other issue that can occur with categorization happens when the Content Filtering subscription expires. When the content filtering subscription expires all categories are set to none. Depending on how this policy is configured, this could have some dramatic results. Click the following link to see examples of how this change will affect your policy:
What happens when my content filtering database expires?
Feedback
