"Missing category list for URL" error using Customer-specific URL categories

Products

• Messaging Gateway

Issue/Introduction

Customer-specific URL categories have been configured but do not appear to be operating as expected. Reviewing the Brightmail Engine logs shows many NOTICE level messages similar to the following:

2022-02-11T10:23:59+05:30 (NOTICE:7131.1540921088): [86010] [ <[email protected]>] UrlAnalyzer: Missing category list for URL 'https://www.example.com' in file 'URL only'.  Original JSON: '[{"uri":"https://www.example.com","risk_level_query_errors":["stargate_status_not_available_in_local_database_cloud_not_attempted"],"query_errors":["stargate_status_not_available_in_local_database_cloud_not_attempted"]}]'.

Environment

Release : 10.7.5

Cause

The Brightmail Engine configuration file containing the customer-specific url category has been loaded by the SMG scanner but the URL category list configuration file is empty most likely due to a failure in propagating configuration from the SMG Control Center.

Resolution

This issue can be resolved by reinitializing the communication channel between the SMG Control Center and it's scanners and then forcing the URL categories configuration to be pushed to the scanners as follows:

For each SMG scanner

1. Connect to the admin command line interface via ssh
2. Run `service agent restart` (this will not affect email processing)

On the SMG Control Center

1. Go to Status > Message Audit Logs and perform a message audit log query. It doesn't matter what the query is, only that it is for "All Scanners"
2. Go to Spam > Customer Specific URL Category Settings
3. Either add or remove a URL category from the category list
4. Click Save

Each of these steps must be performed in the order given. This will first reset the Control Center / Scanner communication channels to ensure that it is operating as expected and then force the propagation of a new URL category configuration.