Configuring Mail Proxies with the PGP Encryption Server (Symantec Encryption Management Server)
search cancel

Configuring Mail Proxies with the PGP Encryption Server (Symantec Encryption Management Server)

book

Article ID: 181072

calendar_today

Updated On:

Products

Encryption Management Server Gateway Email Encryption PGP Key Management Server PGP Key Mgmt Client Access and CLI API PGP Command Line PGP SDK Desktop Email Encryption Drive Encryption Endpoint Encryption File Share Encryption

Issue/Introduction

Mail proxies control how the PGP Encryption Server (Symantec Encryption Management Server) handles the email traffic in your environment.  

When an email comes into SEMS, it will determine where the traffic came from and where it is going using the mail proxies and processes the email correctly.  SEMS uses a proprietary and RFC-compliant mail service to handle all mail called pgpproxyd and this has shown excellent performance and processing.  One reason we call this service pgpproxyd is because SEMS is essentially a "proxy" service, in which a message is received and simply passed to the next outbound hop.  When it does this, the next hop must be available as would be expected for a proxy service.

When the mail proxies for Symantec Encryption Management Server are properly configured, the server can then encrypt and decrypt emails with both SMIME Encryption, PGP Key Encryption, as well as Web Email Protection (Secure Login), and PDF Messenger (Secure PDF).  The Symantec Encryption Management Server will automatically decrypt all email coming in when placed in the mailstream.

Email Encryption will happen automatically based on the rules configured for mail.

The Mail Proxy does not apply to messages generated by Encryption Management Server itself as these take advantage of the sendmail service also available on the SEMS. For example:

  • Daily Status email messages.
  • Web Email Protection and PDF Email Protection notifications.
  • Bounce notifications.



The routing of the above email categories is configured under Mail / Mail Routes in the administration console.

In the Mail / Proxies page of the administration console you can create new proxies and edit existing proxies.

Please see the section Creating New or Editing Existing Proxies in the Symantec Encryption Management Server 10.5 Administrator's Guide for details about mail proxies.


See also the following article for Troubleshooting help with the proxies:

153426 - Troubleshooting: Mailflow with Symantec Encryption Management Server (PGP Server)

Environment

Symantec Encryption Management Server 3.4.2 and above.

Resolution

Just as the name implies and what was mentioned earlier, the SEMS "Proxies" are going to behave as a proxy.  Whenever it is placed in the mailflow, it does not accept the full message and quit the connection before handling the message, it will accept the connection from the sending hop, then establish a connection to the next hop, and once that happens, it will "proxy" the message.  If SEMS is sitting between two mail hops, it must be able to communicate to **both** servers in order for the connection to be successful.

Consider the following outbound flow:

Mailserver --> SEMS --> MTA --> Internet

In the above outbound mail flow example, SEMS will receive the message from the Mail server, and then it will immediately establish a connection to the MTA.  If it is unable to establish a connection to the MTA, the SEMS will refuse to process the message from the mailserver further.  Once the SEMS is able to establish a connection to the MTA, it will then accept the message from the Mailserver and then send the message on out in the next hop.

Table of Contents

 

Section 1 of 6: Creating New or Editing Existing Proxies

You can add or edit the following types of proxies:

  1. SMTP. The SMTP protocol is available for internal or gateway placements. With an internal placement, you can only create or edit an Outbound SMTP proxy. With a gateway placement, you can create or edit an Outbound, Inbound, or Unified SMTP proxy. 
  2. POP. The POP protocol is available only for internal placements. The POP protocol is used by email clients to retrieve email messages from a mail server.
  3. IMAP. The IMAP protocol is also available only for internal placements. The IMAP protocol is also used by email clients to retrieve email messages from a mail server.

Important Note: For Proxy Configurations where the next hop for both inbound and outbound are the same host, the best recommendation is to ensure the PGP server has a separate NIC/IP Address for inbound and a separate NIC/IP for Outbound.  You can then configure the proxy accordingly and then the PGP server will understand the direction of the mail flow.

 

Section 2 of 6: Creating or Editing an Outbound SMTP Proxy

An Outbound SMTP proxy can be configured for either an internal placement or a gateway placement of your Encryption Management Server.
In an internal placement, the Outbound SMTP proxy proxies messages that are sent by your internal email users to the local mail server for delivery to the intended recipient.
In a gateway placement, the Outbound SMTP proxy proxies messages that are sent by your outward-facing mail server to the Internet on the way to the intended recipient.
To create or edit an Outbound SMTP proxy
1.To edit an existing Outbound SMTP proxy, click the name of the proxy you want to edit in the Proxy column of the Mail Proxies page.
The Edit Mail Proxy page appears.
2.If you are creating a new Outbound SMTP proxy, click Add Proxy on the Mail Proxies page, select SMTP from the Protocol menu, and then select Outbound from the SMTP Proxy Type in the Proxy Peer section.
The Add Mail Proxy: SMTP page appears.
3.In the Connector 1 field, in the Local Connector section, select the interface for the local connector for this proxy from the drop-down menu.
The interfaces available are those configured on the Network Settings page (System > Network).
4.In the Port field, select the appropriate port.
The default port for SMTP is 25. The default for SMTPS (secure SMTP) is 465.
5.In the Security menu, select one of the following:
·SSL. Uses SSL to protect the connection between the email client and Encryption Management Server.
·STARTTLS Allow. Allows the security of the connection to be upgraded to TLS through negotiation when communications begin. The email client must support STARTTLS for the upgrade to occur.
·STARTTLS Disable. STARTTLS is not allowed for this connection.
·STARTTLS Require. Requires that the connection be secured by TLS. Select this option if you are confident that all email clients connecting to this local connector support upgrading the security to STARTTLS.
6.Click Restrict Access to enhance the security of this local connector by restricting access by IP address. This step is optional depending on your environment.
7.On the Access Control for Connector dialog box, select the Enable Access Control for Connector check box. This step is optional depending on your environment.
8.Select Hostname/IP or IP Range. This step is optional depending on your environment.
·In the Hostname/IP field, type a hostname or IP address, then click Add. What you type here appears in the Block or Allow field below. If you type a hostname such as example.com, the name resolves to an IP address.
·In the IP Range fields, type starting and ending IP addresses of an IP address range, then click Add. What you type here appears in the Block or Allow field below.
·In the Block or Allow field, select Block these addresses or Allow only these addresses, as appropriate, for the IP addresses or ranges in the box below.
9.To remove an IP address or range from the box, select it, and then click Remove.
10. Click Save when you have configured the appropriate access control restrictions.
The Access Control for Connector dialog box disappears.
11. In the Proxy Peer section, select one of the following:
·Send mail directly to recipient mailserver. When selected, the outgoing email messages coming from your internal email users are sent to the recipient mail server after processing by Encryption Management Server as per the appropriate policies.
·Proxy mail to SMTP server. When selected, the outgoing email messages from your internal email users are sent to the device you specify after processing by Encryption Management Server as per the appropriate policies.
12. If you select Proxy mail to SMTP server, in the Hostname field, type the hostname or IP address of the device you want outgoing email messages to be sent to after processing by Encryption Management Server.
13. In the Port field, select the appropriate port. The default port for SMTP is 25. The default port for SMTPS (secure SMTP) is 465. The port number automatically changes based on your selection from the Security menu.
14. In the Security menu, select SSL, STARTTLS Attempt, STARTTLS Disable, or STARTTLS Require. These are the same options available for the Security menu in the Local Connector section.
15. Click Save.
 

Section 3 of 6: Creating or Editing an Inbound SMTP Proxy

The Inbound SMTP proxy processes email traffic coming into your network from the Internet. An Inbound SMTP proxy can be configured only for a Encryption Management Server in a gateway placement.
To create or edit an Inbound SMTP proxy
1.To edit an existing Inbound SMTP proxy, click the name of the proxy you want to edit in the Proxy column of the Mail Proxies page.
The Edit Mail Proxy page appears.
2.To create a new Inbound SMTP proxy, click Add Proxy on the Mail Proxies page, select SMTP from the Protocol menu, and then select Inbound from the SMTP Proxy Type in the Proxy Peer section.
The Add Mail Proxy: SMTP page appears.
3.In the Connector 1 field, in the Local Connector section, select the interface for the local connector for this proxy from the drop-down menu.
The interfaces available are those configured on the Network Settings page (System > Network).
4.In the Port field, select the appropriate port.
The default port for SMTP is 25 and for SMTPS (secure SMTP) is 465.The port number automatically changes based on your selection from the Security menu.
5.In the Security menu, select one of the following:
·STARTTLS Allow. Allows the security of the connection to be upgraded to TLS throughnegotiation when communications begin. The external MTA must support STARTTLS for the upgrade to occur.
·STARTTLS Disable. STARTTLS is not allowed for this connection.
·STARTTLS Require. Requires that the connection be secured by TLS. Select this option if you are confident that all the devices connecting to this local connector support upgrading the security to STARTTLS.
·SSL. Uses SSL to protect the connection between the external MTA sending and Encryption Management Server.
6.Click Restrict Access to enhance the security of this local connector by restricting access by IP address. This step is optional depending on your environment.
7.On the Access Control for Connector dialog box, select the Enable Access Control for Connector check box. This step is optional depending on your environment.
8.Select Hostname/IP or IP Range. This step is optional depending on your environment.
·In the Hostname/IP field, type a hostname or IP address, and then click Add. What you type here appears in the Block or Allow field below. If you type a hostname such as example.com, the name resolves to an IP address.
·In the IP Range fields, type starting and ending IP addresses of an IP address range, then click Add. What you type here appears in the Block or Allow field below.
·In the Block or Allow field, select Block these addresses or Allow only these addresses, as appropriate, for the IP addresses or ranges in the box below.
9.To remove an IP address or range from the box, select it then click Remove.
10. Click Save when you have configured the appropriate access control restrictions.
The Access Control for Connector dialog box disappears.
11. In the Mailserver field, in the Proxy Peer section, in the Hostname field, type the hostname or IP address of the device you want incoming email messages to be sent to after processing by Encryption Management Server.
Under most circumstances, this should be your outward-facing mail server.
12. In the Port field, select the appropriate port. The default port for SMTP is 25 and for SMTPS (secure SMTP) is 465. The port number automatically changes based on your selection from the Security menu.
13. In the Security menu, select SSL, STARTTLS Attempt, STARTTLS Disable, or STARTTLS Require. These are the same options available for the Security menu in the Local Connector section.
14. Click Save.
 

Section 4 of 6: Creating or Editing a Unified SMTP Proxy

 
The Unified SMTP proxy is a single proxy that includes the properties of both the Inbound SMTP proxy and the Outbound SMTP proxy. In fact, you can individually configure one Inbound and one Outbound SMTP proxies and achieve the same result as with the Unified SMTP proxy.
The Unified SMTP proxy can only be configured for a Encryption Management Server in gateway placement.
With the Unified SMTP proxy, all mail traffic arrives on the same local connectors. This means that you do not need a second IP address for your Encryption Management Server, which you would need if you created separate Inbound and Outbound SMTP proxies.
The Encryption Management Server checks the source IP addresses of all incoming mail traffic on its local connectors and decides the traffic fits one of these two categories:
·The mail traffic is coming from an IP address on the Designated Source IPs list. This traffic is, therefore, an outbound traffic coming from an internal mail server, and is processed as such. Messages are encrypted and/or signed as per the applicable policy, but not decrypted or verified.
·The mail traffic is coming from an IP address not on the Designated Source IPs list. This traffic is thus inbound traffic coming from the Internet, and is processed as such. Messages are decrypted and verified, but not encrypted or signed.
To create or edit a Unified SMTP proxy
1.To edit an existing Unified SMTP proxy, click the name of the proxy you want to edit in the Proxy column of the Mail Proxies page.
The Edit Mail Proxy page appears.
2.If you are creating a new Unified SMTP proxy, click Add Proxy on the Mail Proxies page, select SMTP from the Protocol menu, and then select Unified from the SMTP Proxy Type in the Proxy Peer section.
The Add Mail Proxy: SMTP page appears.
3.In the Connector 1 field, in the Local Connector section, select the interface for the local connector for this proxy from the drop-down menu.
The interfaces available are those configured on the Network Settings page (System > Network). If you want more interfaces to be available for your proxies, you need to configure them on the Network Settings page.
4.In the Port field, select the appropriate port.
The default port for SMTP is 25 and for SMTPS (secure SMTP) is 465.
The port number automatically changes based on your selection from the Security menu.
5.In the Security menu, select one of the following:
·STARTTLS Allow. Allows the security of the connection to be upgraded to TLS through negotiation when communications begin. The external MTA must support STARTTLS for the upgrade to occur. The default port is 25.
·STARTTLS Disable. STARTTLS is not allowed for this connection. The default port is 25.
·STARTTLS Require. Requires that the connection be secured by TLS. Select this option if you are confident that all devices connecting to this local connector support upgrading the security to STARTTLS. The default port is 25.
·SSL. Uses SSL to protect the connection between the external MTA and Encryption Management Server. The default port is 465.
6.Click Restrict Access to enhance the security of this local connector by restricting access by IP address. This step is optional depending on your environment.
7.On the Access Control for Connector dialog box, select the Enable Access Control for Connector check box. This step is optional depending on your environment.
8.Select Hostname/IP or IP Range. This step is optional depending on your environment.
·In the Hostname/IP field, type a hostname or IP address, and then click Add. What you type here appears in the Block or Allow field below. If you type a hostname such as example.com, the name will be resolved to an IP address.
·In the IP Range fields, type starting and ending IP addresses of an IP address range, then click Add. What you type appears in the Block or Allow field below.
·In the Block or Allow field, select Block these addresses or Allow only these addresses, as appropriate, for the IP addresses or ranges in the box below.
9.To remove an IP address or range from the box, select it then click Remove.
10. Click Save when you have configured the appropriate access control restrictions.
The Access Control for Connector dialog box disappears.
11. In the Designated Source IPs list, add the internal mail server(s) that sends mail traffic to Encryption Management Server that is outbound for the Internet.
12. To add the IP address of a mail server, click the plus sign icon, type the IP address, then click Save.
The Unified SMTP proxy considers all mail traffic coming from IP addresses on this list to be outbound for the Internet, and processes it accordingly.
13. Select one of the following:
·Send mail directly to recipient mailserver. When selected, the outgoing email messages coming from your internal email users will be sent to the recipient mail server after processing by the Encryption Management Server per the appropriate policies.
·Send all outbound mail to relay. When selected, the outgoing email messages from your internal email users will be sent to the device you specify after processing by the Encryption Management Server per the appropriate policies.
14. If you selectSend all outbound mail to relay, in the Hostname field, type the hostname or IP address of the device you want outgoing email messages to be sent to after processing by Encryption Management Server.
15. In the Port field, select the appropriate port. The default port for SMTP is 25. The default port for secure SMTP is 465. The port number automatically changes based on your selection from the Security menu.
16. In the Security menu, select SSL, STARTTLS Attempt, STARTTLS Disable, or STARTTLS Require. These are the same options available for the Security menu in the Local Connector section.
17. In the Mailserver field, for Hostname, type the hostname or IP address of the device you want incoming email messages to be sent to after processing by Encryption Management Server.
18. Under most circumstances, this should be your outward-facing mail server.
19. In the Port field, select the appropriate port. The default port for SMTP is 25 and for SMTPS (secure SMTP) is 465. The port number automatically changes based on your selection from the Security menu.
20. In the Security menu, select SSL, STARTTLS Attempt, STARTTLS Disable, or STARTTLS Require. These are the same options available for the Security menu in the Local Connector section.
21. Click Save.
 

Section 5 of 6: Creating or Editing a POP/IMAP Proxy

To create or edit a POP/IMAP proxy
1.To edit an existing POP or IMAP proxy, click the name of the proxy you want to edit in the Proxy column of the Mail Proxies page.
The Edit Mail Proxy page appears.
2.To create a new POP or IMAP proxy, click Add Proxy on the Mail Proxies page and select POP or IMAP, as appropriate, from the Protocol menu.
The Add Mail Proxy: POP or IMAP page appears.
3.In the Connector 1 field, in the Local Connector section, select the interface for the local connector for this proxy from the drop-down menu.
The interfaces available are those configured on the Network Settings page (System > Network).
4.In the Port field, select the appropriate port.
The default for POP is 110 and for IMAP is 143. The default for POPS (secure POP) is 995 and for IMAPS (secure IMAP) is 993.
5.In the Security menu, select one of the following:
·STARTTLS Allow. Allows the security of the connection to be upgraded to TLS through negotiation when communications begin. The email client must support STARTTLS for the upgrade to occur.
·STARTTLS Disable. STARTTLS is not allowed for this connection.
·STARTTLS Require. Requires that the connection is secured by TLS. Select this option if you are confident that all the email clients connecting to this local connector support upgrading the security to STARTTLS.
·SSL. Uses SSL to protect the connection between the email client and Encryption Management Server.
6.Click Restrict Access to enhance the security of this local connector by restricting access by IP address.
7.On the Access Control for Connector dialog box, select the Enable Access Control for Connector check box.
8.Select Hostname/IP or IP Range.
·In the Hostname/IP field, type a hostname or IP address, and then click Add. What you type here appears in the Block or Allow field. If you type a hostname such as example.com, the name resolves to an IP address.
·In the IP Range fields, type starting and ending IP addresses of an IP address range, and then click Add. What you type here appears in the Block or Allow field below.
·In the Block or Allow field, select Block these addresses or Allow only these addresses, as appropriate, for the IP addresses or ranges in the box below.
9.To remove an IP address or range from the box, select it, and then click Remove.
10. Click Save when you have configured the appropriate access control restrictions.
The Access Control for Connector dialog box disappears.
11. In the Mail server field, in the Proxy Peer section, type the mail server from which the email clients attempt to retrieve their messages.
This is the mail server from which the email clients retrieve their messages directly, if the Encryption Management Server is not between the flow of email traffic.
12. In the Port field, select the appropriate port.
The default for POP is 110 and for IMAP is 143. The default for POPS (secure POP) is 995 and for IMAPS (secure IMAP) is 993.
The port number automatically changes based on your selection from the Security menu.
13. In the Security menu, select one of the following:
·STARTTLS Attempt. Allows the security of the connection to be upgraded to TLS through negotiation when communications begin. The mail server must support STARTTLS for the upgrade to occur.
·STARTTLS Disable. STARTTLS is not allowed for this connection.
·STARTTLS Require. Requires that the connection be secured by TLS. Select this option if you are confident that the mail server connecting to this local connector supports upgrading the security to STARTTLS.
·SSL. Uses SSL to protect the connection between Encryption Management Server and the mail server.
14. Click Save.







Section 6 of 6: Troubleshooting


There are some cases that may happen intermittently when a next hop, such as an MTA may not always communicate properly with the pgpproxyd service.  In these cases, it is possible to configure the mail proxy to send all outbound messages where the last service to handle the message is "sendmail".  Some third-party mail services may communicate better with sendmail.  In order to configure the proxy, consider the following output setup:
 
Mailserver --> SEMS --> MTA --> Internet
 
In the above scenario, the pgpproxyd service will communicate with the MTA.  If there are some intermittent issues sending mail to the next hop, it may be useful to configure SEMS to use sendmail instead.  The following errors may be observed when this happens:

"error handling SMTP DATA event: corrupt data"
"error handling SMTP DATA event: unknown error"
"Error while processing"
"error handling SMTP DATA event: out of memory"
"SMTP Data ProtocolEvent returning with error -11980 (unknown error)"
"smtpsrv timeout reading command"
 
 
In the SEMS UI, the proxies may be configured as follows:
 
 
As you can see in the example above, the SEMS is sending to "mta.example.com" as the next hop outbound.  If this is done, this will use pgpproxyd and this works in most cases.  If you are seeing some oddities with processing, you can try changing this to use sendmail instead.  Follow the steps below to do this:
 
1. Change the configuration to use the following:

 
 
As you can see we have the next outbound hop configured for "Send mail directly to recipient mailserver".  Obviously the next hop is "mta.example.com" so we cannot leave the configuration like this.  We must now add a mailroute to the proxies so that the next hop will be "mta.example.com".  To do this, click on Mail, Mail Routes, and configure the following:




In the screenshot above, * is used as a wildcard for "All Domains", and for any domain, all mail will be sent to "mta.example.com".  If this same mta.example.com handles all inbound and outbound mail, you can leave it.  If you have a specific inbound hop, such as a mailserver that you need to go to instead, then you need to add another mail route specifically for that mailroute.  


In this example, we'll use "mailserver.example.com" for the inbound mailserver and for the domain, we'll use "example.com".  For the mail routes, we'll configure the following:
 


In the screenshot above this rule will mean that all mail for "example.com" will be sent to "mailserver.example.com".  In this way, you can configure what you need for mail to flow properly. 
 
 
 
 

Note on * Domain Mail Route

If you have a configuration that requires all email to be sent to a specific MTA for outbound email flow, then this rule is useful to ensure **all** email goes to that host.

If you have an MTA with host "mta.example.com", and your mail proxies are configured with mail.example.com for both the inbound and outbound flow, you may still see emails being sent from the PGP server where PGP is trying to connect to the external hosts.  This is because the PGP server is designed to do MX lookups for all domains, but when you add in the mailroute, it will prevent the PGP server from making these MX Lookups.
 
In the scenario where * is the domain and mta.example.com is configured for the mail route, this will also mean that any email to any domain will always go to mta.example.com and will no longer do MX lookups.  While this will avoid sending to external mailservers, it will mean you will need to configure your mta to be able to receive all email from the PGP server.
 
Generally the above configuration is fine when the mta is handling all mail for inbound and outbound to which the PGP server will communicate. 
 

Additional Information

153426 - Troubleshooting: Mailflow with Symantec Encryption Management Server (PGP Server)

150133 - Header and body flags that indicate PGP encrypted email for SPAM filter and mail server configuration

180151 - HOW TO: Create Policy Chains to Set Mail Policy in PGP Server (Symantec Encryption Management Server)

181072 - Configuring Mail Proxies with the PGP Server (Symantec Encryption Management Server)

156100 - Emails going to exception chain on the PGP Server (Symantec Encryption Management Server)

EPG-22595, EPG-23433

Powered by Wolken Software