Troubleshooting Email Encryption or Decryption with PGP Encryption Server (Symantec Encryption Management Server)
Article ID: 254080
Updated On:
Products
Issue/Introduction
The PGP Encryption Server (Symantec Encryption Management Server) has advanced functionality that will allow you to encrypt data in many different ways.
This article will go over the basics in troubleshooting Encryption and some scenarios to be aware of.
Resolution
The PGP server uses two standards to encrypt email:
*PGPMIME
*SMIME
In both of these cases, Public and Private keys are created.
When you need to encrypt, we use the Public portion of the key.
When you need to decrypt, we use the Private portion of the key.
The term "Keypair" denotes that you have both the Public and the Private Portion of the keys.
Public and Private keys have other functions as well.
When you encrypt data, you sometimes need to "sign" it, so that the recipient knows the data did not change in transit.
When you "Sign" email, you will use the Private Key to sign these email messages. The reason you sign with the Private Key is a passphrase is associated to that key and only the owner of that key knows what it is. This establishes "non-repudiation", so that when something is signed, you know exactly who signed it, and with what key. This "Signing" cannot be done w/out having the Private portion of the key and as the owner of the key, only you have that.
Once email is signed, you want to be able to validate if the signature is what you expect it to be as well as if the signature matches the signer. To validate, you need the Public portion of the key. The signature is linked to the public portion so if someone signs an email, you don't need to have the signers Keypair/Private Key to validate the signature. You simply use the Public portion of the key to validate this.
When a PGP Key is generated, a "Key ID" is created, which is a small string of numbers and letters that are unique to only that key.
When an SMIME certificate is generated, a "Fingerprint" is associated to the key, which is also unique to only that SMIME certificate.
In this way, when a key/certificate is generated, you know there is a specific Key ID or Fingerprint associated to it and that you can know you are working with the proper key in question.
With the above understanding in mind, troubleshooting encryption and decryption issues is largely related to having the proper keys and knowing the Key IDs\Fingerprints so that you know data is being encrypted to the proper keys and that only the intended recipient, who has the corresponding private key can decrypt.
Troubleshooting:
Table of Contents
- Scenario 1: Make sure the proper key is being used:
- Scenario 2: Make sure you have the keypair to decrypt
- Scenario 3: Make sure you have the password/passphrase
- Scenario 4: Make sure the algorithms and ciphers are compatible
- Scenario 5: General Compatibility
- Scenario 6: Look to see if something else is interfering with the Encryption or Decryption
Scenario 1: Make sure the proper key is being used:
When an email needs to be encrypted, you need to have the public key for the recipient. When you start working with the entity, call them on the phone and ask them the Key ID so that you know you have the actual key.
For SMIME, the public certificate is commonly attached to the email from the sender so that you can assume the SMIME certificate is proper. You still the ability to look at the Fingerprint and determine if you have the proper key if you are still in doubt.
Scenario 2: Make sure you have the keypair to decrypt
If you have received an encrypted email, but it is not decrypting, find out if the sender may have encrypted to the wrong key. Making a phone call can usually establish this is the case.
Then ensure the proper key gets used for future encryption.
Scenario 3: Make sure you have the password/passphrase
If someone encrypts an email to you, but you don't know the passphrase, you won't be able to decrypt.
In these cases, keep trying, but if you still can't, then you may need to generate a new key and exchange.
The sender will need to encrypt to the new key going forward.
Scenario 4: Make sure the algorithms and ciphers are compatible
As mentioned, PGP Server uses "PGPMIME" or "SMIME" as the encryption standards.
Within these standards are various algorithms. Some keys have a certain set of algorithms that are supported, and others are not.
If you have the proper key, and you know the passphrase, but still can't decrypt, look at the algorithms.
Scenario 5: Validity of the SMIME Root
If you are encrypting to SMIME certificates, you will need to trust the Root Certificate Authority who signed the user's SMIME cert.
Certificate Authorities, such as SMIME offer their own way to generate SMIME certs, and when these trusted CAs are used, it makes it easy to exchange the data.
If you work with an entity who has their own "Internal CA", this means that the Root may not be trusted externally, so you may need to manually trust the Root Certificate before encryption can work.
Scenario 5: General Compatibility
There are sometimes cases that encryption may not happen due to certain requirements for the keys. One situation where encryption may not take place is a particular vendor may require an attribute that may not exist on the keys that may make the key look unusable, or may not appear as valid. Work with that vendor to gather specific requirements and if needed, reach out to Symantec Encryption Support to ensure these requirements can be met.
Using tools, such as openssl can help parse the attributes of the cert, and the vendor may be able to identify if certain values are missing:
openssl asn1parse -in Users-Cert.pem
(Look for "Null" values to indicate potential attributes missing)
You can also use OpenSSL to validate certs when linked to their roots:
openssl verify -CAfile Root-or-Intermediate-CA-Signer.pem Users-Cert.pem
(Look to see if the validation works or not):
Successful Example:
Users-Cert.pem: OK
Failed Example:
error 7 at 0 depth lookup:certificate signature failure
As a workaround, have the recipient encrypt to the cert without validation. If this is not possible, please reach out to Symantec Encryption Support for further guidance and reference ID EPG-28318 and we will assist further.
Scenario 6: Look to see if something else is interfering with the Encryption or Decryption
As security is a top priority for any environment, other security applications may get in the way of decryption. It's a good idea to look at other security applications and ensure they are added to a "Safe List" or "Exclusion" so that the PGP Encryption/Decryption and perform properly.
Additional Information
EPG-28318
Feedback
