BUG REPORT: Cached keys with multiple "Signature Packets" and increased size might cause slow mail processing

book

Article ID: 158478

calendar_today

Updated On:

Products

Encryption Management Server Gateway Email Encryption

Issue/Introduction

Cached keys under "Keys" - "Key Cache" might increase in size over time which can cause slow mail processing.

This might cause SMTP connections to time out.

Depending on your configuration the following error messages might be logged in the proxy logs.

- pgpproxy: not enough contiguous memory to read in message error=-11999 (out of memory)    
- pgpproxy: Error processing SMTP message, awaiting next client command. (-11980).
- pgpproxy: unable to send mail transaction data to server error=-11990
- pgpproxy: error reading/processing message error=-11990 (read failed)
- SMTP Data ProtocolEvent returning with error -11980 (unknown error)
- error handling SMTP DATA event: out of memory
- error handling SMTP DATA event: read failed
   

Additionally your CPU is constantly overloaded while the errors come up in the logs.

 

Cause

When sending a S/MIME signed mail, the x509 signing certificate is attached to the mail. Per default Symantec Encryption Management Server will harvest such x509 certificates from the mail flow and store them as cached keys.

The defect is caused by malformed keys with multiple "Signature Packets" in the key cache of the server. As a consequence, your cached key(s) will grow in size and may cause the database to be slow when reading/writing such keys from/to the database. SMTP connections might take much longer and can eventually time out depending on the configured mail server timeouts.

To verify if a key is affected PGP Commandline, GnuPG or pgpdump can be used to dump the large cached key. Affected keys will be large in size and contain multiple "signature packets".

Example commands:

# pgp --list-packets cached_key.asc
# gpg --list-packets cached_key.asc
# pgpdump cached_key.asc

Example output of PGP Commandline containing 4 additional signature packets with unknown algorithm 100:

Old: Signature Packet(tag 2)(329 bytes)
	Ver 4 - new
	Sig type - Generic certification of a User ID and Public Key packet(0x10).
	Pub alg - unknown(pub 100)
	Hash alg - SHA1(hash 2)
	Hashed Sub: vendor specific(sub 100)(187 bytes)
	Unrecognized vendor subpacket
	Hash left 2 bytes - 26 bb 
	Unknown signature(pub 100)
Old: Signature Packet(tag 2)(329 bytes)
	Ver 4 - new
	Sig type - Generic certification of a User ID and Public Key packet(0x10).
	Pub alg - unknown(pub 100)
	Hash alg - SHA1(hash 2)
	Hashed Sub: vendor specific(sub 100)(187 bytes)
	Unrecognized vendor subpacket
	Hash left 2 bytes - 0e 27 
	Unknown signature(pub 100)
Old: Signature Packet(tag 2)(329 bytes)
	Ver 4 - new
	Sig type - Generic certification of a User ID and Public Key packet(0x10).
	Pub alg - unknown(pub 100)
	Hash alg - SHA1(hash 2)
	Hashed Sub: vendor specific(sub 100)(187 bytes)
	Unrecognized vendor subpacket
	Hash left 2 bytes - 79 dc 
	Unknown signature(pub 100)
Old: Signature Packet(tag 2)(329 bytes)
	Ver 4 - new
	Sig type - Generic certification of a User ID and Public Key packet(0x10).
	Pub alg - unknown(pub 100)
	Hash alg - SHA1(hash 2)
	Hashed Sub: vendor specific(sub 100)(187 bytes)
	Unrecognized vendor subpacket
	Hash left 2 bytes - 2d 6b 
	Unknown signature(pub 100)

Resolution

This issue is fixed in the following release:

  • 3.3.2 MP1

This version/Maintenance Pack is available for download via your account on Symantec File Connect.


The following is known as a workaround for the issue until the maintenance pack or version that contains the fix is released:

- Remove the questionable keys from the key cache or purge the whole key cache.
- Import the sender certificate manually as an "External User". Keys that are imported to the external users won't be harvested in the key cache anymore. It's recommended to import the original certificate from the sender after deleting the key from the key cache. Please refrain from importing the malformed keys from the key cache as the key might already contain unwanted signature packets.

If you want to have this issue verified or you are still unsure which keys to remove, please contact the Symantec Technical Support.


Applies To

Symantec Encryption Management Server or PGP Universal Server in gateway mode.