Unable to import certificate into SMTP Prevent keystore: "Input not an X.509 certificate"

Products

• Data Loss Prevention Network Prevent for Email

Issue/Introduction

You are setting up TLS encryption for an SMTP Prevent server and need to import the public key from the downstream MTA. When issuing the keytool commands as described in the documentation, you receive a Java exception showing that the input file is not an X.509 certificate.

Environment

DLP 15.X

Cause

Certificate is not in the correct format supported by our keystore. Any certificate that we add to the Network Prevent for Email keystore must be an X.509 certificate in Private Enhanced Mail (.pem) Base64-encoded Distinguished Encoding Rules (DER) certificate format, enclosed within -----BEGIN CERTIFICATE----- and -----END CERTIFICATE----- strings in the certificate file.

Resolution

Review the file you are attempting to import. If it contains a BEGIN CERTIFICATE and END CERTIFICATE line near the top and bottom of the file, it is most likely in PEM format, and should be converted to the binary DER format for use with keytool.

You can use OpenSSL (installed by default on Linux servers, but not on Windows) to check that the certificate is valid, while it is in the PEM format. Substitute the appropriate path and filename for the examples below:

openssl x509 -in mycert.txt -text

Examine the output to be sure that the certificate shows a public key signed by the correct authority. If the certificate is valid and intact, OpenSSL can also convert the certificate to DER format using the following command (again, substitute path and filename where applicable):

openssl x509 -in mycert.txt -inform PEM -out mycert.cer -outform DER

You can also convert a .pfx OR .p7b using following command and then import the .pem cert:

openssl pkcs12 -in mycert.pfx -out mycert.pem -nodes

You can then continue with the import commands as shown in the MTA Integration Guide.