Upgrading PGP Encryption Management Server with the Web Email Protection Service Enabled (Symantec Encryption Management Server)

Products

• Gateway Email Encryption
• Encryption Management Server
• Desktop Email Encryption
• Drive Encryption
• Endpoint Encryption
• File Share Encryption
• PGP Command Line
• PGP Key Management Server
• PGP Key Mgmt Client Access and CLI API
• PGP SDK

Issue/Introduction

When upgrading a PGP Encryption Management Server (Symantec Encryption Management Server) as a single node, downtime may be needed.  To avoid this, having cluster nodes that manage the same services will help reduce this likelihood.

When upgrading a PGP Server cluster, it is possible to avoid downtime. For example, with a two-member cluster comprising member1 and member2:

1. Ensure that member1 is processing all email messages and that all PGP Encryption Desktop (Symantec Encryption Desktop) clients are pointing to it. This may require updating DNS or reconfiguring the load balancer.
2. Upgrade member2.
3. Update DNS or the load balancer to ensure that member2 is processing all email messages and that all PGP Desktop clients are pointing to it.
4. Upgrade member1.
5. Optionally, update DNS or the load balancer to ensure that member1 is processing all email messages and that all PGP Desktop clients are pointing to it.

If Web Email Protection or PDF Email Protection with an option such as secure replies is enabled, downtime is generally recommended. Consider what happens when the above steps are used to upgrade a two member cluster running Web Email Protection:

1. Ensure that member1 is processing all email messages and that all PGP Desktop clients and/or Web Email Protection users are pointing to it.
2. Upgrade member2.
3. While member2 is being upgraded, the following may occur on member1:
   • New Web Email Protection accounts are created.
   • New inbound Web Email Protection messages are created.
   • New Web Email Protection replies are sent by Web Email Protection users.
4. After member2 has been upgraded, any new Web Email Protection related data will not replicate to member1 because they are running different releases.
5. Ensure that member2 is processing all email messages and that all PGP Desktop clients and/or Web Email Protection users are pointing to it.
6. Web Email Protection users may not be able to log in to member2 because their account was created while member2 was being upgraded and therefore the account exists only on member1.
7. Mail sent to Web Email Protection users while member2 was being upgraded will not exist on member2 and therefore users will be unaware of it.
8. Upgrade member1.
9. While member1 is being upgraded, the Web Email Protection related data described above will be added only to member2.
10. After member1 is upgraded, replication will synchronize the Web Email Protection data around cluster. Full synchronization can only be expected after the overnight "incremental" replication task has run. By default, this task starts at 11pm and runs for a maximum of 4 hours. Depending on the number of changes that need synchronizing, 4 hours may be insufficient to complete the task.

As can be seen, at the point where all cluster members have been upgraded, there may be significant differences in the Web Email Protection data between cluster members. The earliest that full synchronization can be expected is 3am on the day following the upgrade but several days may be required for full synchronization.

If something goes wrong during the upgrade or if replication does not work properly after the upgrade, there is a risk that Web Email Protection data will be inconsistent between cluster members for an extended time and in extreme cases there is a risk of complete data loss. These are risks that most organizations are unwilling to take.

 

Important Tip for WEP to Reduce the need for downtime: Due to some of the operations during these types of upgrades, it is recommended to enable WEP on all nodes for the PGP Server to be able to operate one node at a time.  In particular, each PGP Server can have the WEP service disabled, but this usually means that messages will not reside on each of the nodes, making downtime more likely.  If you have the WEP service enabled on all nodes, as part of the migration, you can have WEP handle the service until one node is migrated. 

Then point traffic to the migrated node, and then you migrate the previous node.  Having the WEP service enable don all nodes also makes replication rings more straightforward.

 

Environment

PGP Encryption Management Server release 10.5 and above.

Resolution

To avoid the risks described above, some downtime will be necessary:

1. Ensure that member1 is processing all email messages and that all Encryption Desktop clients and/or Web Email Protection users are pointing to it.
2. Block inbound HTTPS traffic from accessing member1 over the Internet or pause the Web Email Protection service from the management console by navigating to Services / Web Email Protection and clicking on the Pause button. This will prevent Web Email Protection users from logging in.
3. Stop routing SMTP traffic to member1.
4. Upgrade member1.
5. Start routing SMTP traffic to member1 and allow Web Email Protection users to connect over HTTPS.
6. Upgrade member2.
7. Once member2 is upgraded, replication will synchronize the Web Email Protection data. Do not point SMTP or HTTPS traffic to member2 until at least 3am the day following the upgrade.