Transport Layer Security (TLS) certificate validation verifies both Certificate Authority (CA) signature and hostname match

Products

• Messaging Gateway

Issue/Introduction

When Messaging Gateway (SMG) is configured to require TLS delivery from a specific domain and to verify the certificate, SMG is not accepting delivery from the sender even when they have a valid TLS certificate.

• Symantec Messaging Gateway (SMG) Transport Layer Security (TLS) settings for a domain are set to "Require TLS and verify certificate", but TLS connections fail.
• Mail Exchange (MX) host has a signed TLS certificate.

2010 Aug 17 19:01:11 GMT (debug) ecelerity: [6390] Host Name: mx1.example.com 
2010 Aug 17 19:01:11 GMT (debug) ecelerity: [6390] Host SSL certificate Subject: /C=US/ST=State/L=Locality/O=Company Co/CN=server1.example.com
2010 Aug 17 19:01:11 GMT (debug) ecelerity: [6390] Subject Common Name: server1.example.com 
2010 Aug 17 19:01:11 GMT (info) ecelerity: [6390] Subject Common Name does not match host name 
2010 Aug 17 19:01:11 GMT (info) ecelerity: [6390] DNS Subject Alternative Name does not match host name

Cause

The SMG TLS certificate validation operates in this order:

1. The signing chain for the server certificate is valid
2. The certificate is not out of date
3. The signing certificate authority is trusted by the Brightmail Gateway
4. The Domain Name System ( DNS ) hostname matches the certificate subject common name or subject alternative name (SAN)

If the DNS hostname referred to by the domain's MX records do not match the common name (CN) attribute of the certificate Subject or Subject Alternative Name then certificate validation will fail and a secure connection cannot be established. 

Resolution

This may be resolved if you modify the TLS policy for the domain to "Require TLS and don't verify certificate".

1. Log into the Control Center as an administrator
2. Navigate to the Protocols > Domains page
3. Select the domain you wish to update and click the "Edit" button
4. On the "Delivery" tab, select "Require TLS and don't verify certificate"
5. Save the changes to the domain.