Inbound S/MIME messages fail to be decrypted if Encryption Management Server cannot make outbound HTTP connections
search cancel

Inbound S/MIME messages fail to be decrypted if Encryption Management Server cannot make outbound HTTP connections

book

Article ID: 171558

calendar_today

Updated On:

Products

Encryption Management Server Gateway Email Encryption

Issue/Introduction

If the PGP Encryption Server (Symantec Encryption Management Server) cannot make outbound HTTP connections, an inbound message that is S/MIME signed and/or S/MIME encrypted, is deferred. The sending mail server therefore keeps trying to send the message. This occurs even though the inbound message is successfully processed by Encryption Management Server and successfully passed to its Inbound mail proxy.

The result is that the recipient receives the same message multiple times.

In a configuration such as this:

Internet -> SMTP mail server -> PGP Encryption Server -> Microsoft Exchange Server

This error appears in the PGP Encryption Server mail log. The message is proxied successfully to the Exchange Server but the transmission channel from the SMTP mail server is not closed properly:
2018/04/20 15:28:36 +01:00  NOTICE pgp/messaging[25146]:       SMTP-00000: passing through unmodified
2018/04/20 15:28:36 +01:00  ERROR  pgp/messaging[25146]:       SMTP-00000: error handling SMTP DATA event: write failed
2018/04/20 15:28:37 +01:00  ERROR  pgp/messaging[25146]:       SMTP-00000: pgpproxy: error reading/processing message error=-11989 (write failed)

Environment

Symantec Encryption Management Server 10.5 and above.

Cause

When PGP Encryption Server (Symantec Encryption Management Server) processes an S/MIME signed and/or encrypted message, it checks with the Certificate Authority that issued the certificate whether the certificate used to sign and/or encrypt the message is revoked. Revoked certificates are invalid.

There are two mechanisms used to check whether S/MIME certificates are revoked:

  1. CRL (Certificate Revocation List). The list is downloaded from a remote host over HTTP or LDAP. However, LDAP is not supported by most Certificate Authorities. If HTTP is used, Encryption Management Server prior to release 10.5.1 can only process CRL files that are under 1 MB in size. If the CRL file is over 1 MB, LDAP will be attempted if it is included in the CRL Distribution Points attribute of the certificate. Starting with release 10.5.1, the default limit is 5 MB but this can be raised to 20 MB.
  2. OCSP (On-line Certificate Status Protocol). The service runs on a remote host and Encryption Management Server connects to it over HTTP. The address of the OCSP service is contained in the Authority Information Access attribute of the certificate.

If the PGP Encryption Server cannot make outbound HTTP connections it causes problems with S/MIME mail processing.

Resolution

There are two main solutions to this issue:

  1. Ensure that the PGP Encryption Server can make outbound HTTP and LDAP connections. This is the preferred solution because it is clearly best practice to check whether S/MIME certificates are revoked and this can only be done using outbound HTTP or LDAP connections. Permitting only outbound LDAP will usually not be sufficient because only a minority of Certificate Authorities support LDAP.
  2. If it is not possible to permit the PGP Encryption Server to make outbound HTTP connections, configure it to check certificate revocation status using only CRL and not OCSP.
    This configuration option is available only in versions 3.4 MP1 and above (As of this writing, it is recommended to update to 10.5.1 MP2 or above for best operation). This will resolve the issue with email messages not being proxied properly, though clearly certificate revocation status will not be checked.

If the PGP Encryption Server can make outbound HTTP connections, these configuration options are also available:

  1. Check certificate revocation status using the OCSP service first and fall back to accessing the CRL file over HTTP or LDAP.
  2. Check certificate revocation status using only the OCSP service.

Please open a case with Technical Support if you wish to make any of the above configuration changes.

It is important to note that in order for the PGP Encryption Server to be able to complete the certificate revocation check, it must trust the certificates in the certificate chain of the sender's personal certificate. To trust the issuing certificates, please do the following:

  1. From the Administration console, navigate to Keys / Trusted Keys.
  2. Click on the Add Trusted Key button.
  3. Import the relevant root certificate and all intermediate certificates.
  4. Enable the option Trust key for verifying mail encryption keys.
  5. Optionally, enable the option Trust key for verifying SSL/TLS certificates.
  6. Click the Save button.

Additional Information

235862 - Symantec Encryption Management Server unable to process mail when using OCSP

163194 - Symantec Encryption Management Server may encrypt messages to revoked S/MIME certificates if the CRL or OCSP is unavailable

174739 - Encryption Management Server enables the Certificate Revocation Service by default