VMware vCenter in Enhanced Linked Mode pre-changes snapshot (online or offline) best practice

Products

• VMware vCenter Server

Issue/Introduction

When using multiple vCenter Server Appliances (VCSA) in the same Single Sign-on Domain, replicating in Enhanced Linked Mode (ELM), there is high potential of corruption of the domain if snapshots of the appliances are taken while they are in running state. You should only use offline snapshots in ELM deployments, meaning all appliances should be gracefully shut down, and snapshots need to be taken while the VCSAs are in powered off state (at the same time).

Also be aware that if any change must be reverted, you need to restore all of the nodes in the ELM deployment to this offline/consistent snapshot state. Only start powering the restored nodes back on after all of them have been restored from the snapshots.

Doing otherwise can and will introduce inconsistencies between the local VM Directory instances of the embedded platform service controllers, which will prevent the nodes from successfully replicating with each other.

Notes:

• Alternatively you can also use file-based backups instead. For more information about vCenter file-based backup, please see Backup and Restore options in vCenter Server 6.x/7.0.x/8.0.x - Overview.
• In addition, please also see  vCenter Server Appliance Data Integrity Best Practices.
• For more information about Enhanced Linked Mode, see What is vCenter Enhanced Linked Mode in the official vSphere documentation.
  
  

Environment

• VMware vCenter Server Appliance 6.x
• VMware vCenter Server Appliance 7.0.x
• VMware vCenter Server Appliance 8.0.x

Resolution

As stated above, VMware recommends to have offline Snapshots (virtual machine powered off) of all nodes in the same SSO domain, aka running in ELM replication, before any activity that will include changes in the vCenter Server.

Offline snapshots of all nodes in an SSO domain (ELM) are required when any SSO domain update will be performed. This includes but is not limited to:

• vCenter Server Updates (Full Version, Update Release, or Patch Release).
• Using the lsdoctor tool to make any changes.
• Adding a new vCenter Server to an existing SSO domain.
• Retiring a vCenter Server from an existing SSO domain.
• Certificate Replacement (Machine, CA, STS, etc).

Any read only activity does not require offline snapshot. That means the below activities can be performed with NO need for snapshot (Neither online nor offline). 

• vCenter Server / PSC backups.
• Using the lsdoctor tool  with the --lscheck parameter.
• Using the checksts script.

Caution:

• Offline snapshot is a MUST when using lsdoctor tool unless using the --lscheck switch (only) which is used to check for common issues in the lookup service.
• Make sure there is no backup task running before taking the snapshot of the vCenter Server or the PSC.
• Do not take snapshot of the vCenter VM(s) if it’s part of vCenter High Availability (VCHA) configuration. 

Additional Information

• For more information about vCenter updates, see Patching (update) VMware vCenter Server Appliance VCSA and Platform Service Controller PSC through Appliance Management Interface VAMI.
• For more information about the lsdoctor tool, see Using the 'lsdoctor' Tool.
• For more information about the checksts script, see Checking Expiration of STS Certificate on vCenter Servers.
• For more information about joining a vCenter to an existing SSO Domain, see Joining a vCenter Enhanced Linked Mode Domain.
• For more about decommissioning a vCenter Server from existing SSO Domain, see Using the cmsso command to unregister vCenter or PSC from Single Sign-On.
• For more information about vCenter Security Certificates, see vSphere Security Certificates.
• For more information about vCenter High Availability, see FAQ: vCenter High Availability.