Messaging Gateway Brightmail Engine crashes on signal 11 following AV rules update

book

Article ID: 197290

calendar_today

Updated On:

Products

Messaging Gateway

Issue/Introduction

Messaging Gateway's Brightmail engine begins repeatedly crashing on signal 11 and restarting. While the Brightmail engine is restarting, messages accumulate in the inbound and outbound queues.

Cause

On 8/13/2020, a malware and file decomposer update was made available to a small subset of Messaging Gateway customers. This update to the decomposer engine can cause the Brightmail Engine service to crash when processing some email messages.

The Brightmail Engine will automatically restart and if it crashes multiple times on a particular message will quarantine that message in a Bad Message Queue but the crash may be cause by other messages, creating what appears to be crash-restart loop.

Resolution

The malware and decomposer engine which is triggering the issue has been removed from the download repository. The standard LiveUpdate process will automatically roll back any installation which has the newer release on its next update cycle. 

If Liveupdate is not running on the default, 10 minute, update check cycle, deleting the running malware and decomposer engine from the system by hand will also resolve the issue. 

To delete the malware and decomposer engine from SMG and force it to update:

  1. Log into the SMG CLI as `admin`
  2. Run delete avrules

This will remove the running malware and decomposer engine, load the default version, and restart the MTA and Brightmail Engine services. The system will then update to the most recent stable engine automatically.

Rescanning messages marked as bad

Some messages may have been marked as bad after causing the Brightmail Engine to crash three times when attempting to scan them. To release these messages from the Bad Message queue and rescan them:

  1. Log into the SMG command line as admin
  2. List all the bad messages by running
    mta-control  all bad-msg-list
  3. For each message in the list run
    mta-control bad-msg-retry [message id]

For example:

mta-control bad-msg-retry 90/00-04009-9EED53F5