VMware vSAN Upgrade Best Practices
Article ID: 326927
Updated On:
Products
VMware vSAN
Issue/Introduction
Environment
VMware vSAN (All Versions)
Resolution
The success of vSAN Cluster Upgrades depends on the following prerequisite steps to prepare the cluster for the upgrade
as well as adherence to certain recommendations during the upgrade process.
Before starting a vSAN Cluster Upgrade
Ensure that the following requirements are met:
1.) The vSphere Environment is up to date:
- The vCenter Server managing the Hosts must be at an equal or higher version than the ESXi Hosts version it manages. It is advisable to have vCenter and ESXi on matching Versions as doing otherwise can lead to communication issues between the two as per KB vCenter version to ESXi version (318674) (Refer to KBs Build numbers and versions of VMware ESXi/ESX (316595) and Build numbers and versions of VMware vCenter Server (326316) to determine supported vCenter/ESXi version combinations.)
- All hosts should be running the same build of ESXi before vSAN Cluster upgrade is started. Only uniform ESXi Versions across the Cluster will ensure efficient vSAN functionality.
- If the ESXi Versions are not matched, the Hosts should be patched to the same Build before upgrading.
2.) It is a must that all checks in vSAN Skyline Health do show green
Note: The Cluster should not have any failed or absent Disks. Ensure that all vSAN Disks are showing up in the vSAN Disk Management.
3.) The HCL for the vSAN Controller should have a matching Driver/Firmware combination and it should also be supported with the target version of ESXi
The HCL can be verified by checking the driver/firmware version installed on the host with the database listed via vSAN HCL.
4.) There should NOT be any inaccessible vSAN Objects
This can be verified with the vSAN Health Service in vSAN 6.0 and above, or with the Ruby vSphere Console (Attached to this KB)
5.) There should not be any active Resync at the start of the Upgrade process
Some Resync activity is expected during the Upgrade process, as Data needs to be synchronized following Host reboots.
The Administrator must wait till Resync finishes before putting the next Host into Maintenance Mode.
6.) Ensure that there are no known compatibility issues between your current vSAN Version and the desired target vSAN Version
- For information on upgrade requirements, see vSAN upgrade requirements (326926).
- Check the Upgrade path for the compatibility
- If required, update the vSAN cluster to the required Build before undertaking the Upgrade process to avoid compatibility concerns.
ESXi Host Preparation
Ensure you choose the appropriate Maintenance mode option for your Environment:
- Ensure Availability: vSAN allows you to move the Host into Maintenance mode faster than under the Mode "Full Data Migration" and ensures access to the Virtual Machines in the Environment.
- Full Data Migration: vSAN evacuates all data to other Hosts in the Cluster. This Evacuation mode results in the largest amount of data transfer and consumes the most time and resources.
- No Data Migration: vSAN does not evacuate any data from this Host. If you power off or remove the Host from the Cluster, some Virtual Machines might become inaccessible. This is not a safe option to be used
Exit Maintenance Mode and Resync:
When the ESXi Host is upgraded and moved out of Maintenance Mode, a Resync will occur if this took more than 60mins which is the default Resync delay timer. You can see this via the Web client.
Ensure the Resync has been completed before moving on to the next Host.
( A Resync is occurring as the Host that has been updated can now contribute to the vSAN Datastore again.
It is vital to wait till this Resync has been completed to ensure there is no data loss )
For Stretched vSAN clusters with target builds of 7.0 or higher always upgrade the witness host before the physical nodes.
Also, check vSAN Health Service - Stretched cluster - Witness appliance upgrade to vSphere 7.0 or higher with caution (326417) if the Witness is being upgraded from 6.x to 7.x or higher version.
Note: For Target Builds below 7.0 (6.7, 6.5, etc) the Witness node should be upgraded after upgrading the physical Hosts.
After starting the vSAN Cluster Upgrade
After beginning the Upgrade process, there are a few items to keep in mind:
Once you start an Upgrade of a vSAN Cluster make sure to complete the upgrade ASAP preferably within a week's time as mixed versions of ESXi in the same cluster,
especially a difference of major releases, is not a supported configuration and can cause issues such as performance issues and cluster instability.
This is due to having mixed codes talking to each other within the same cluster.
Mixed Versions are ONLY supported during an upgrade which is expected to be completed typically within a 24-48hr period for clusters below 32 hosts.
For large clusters, 32-64 hosts typical upgrade should be completed within 48-72hrs.
Do NOT attempt to Upgrade a Cluster by introducing new Versions to the cluster and migrating workloads.
If introducing new Host(s) mid-cluster upgrade ensure no disk groups are present/created until all Hosts have been upgraded to the same ESXi version to prevent potential vSAN network partitions.
Be sure to complete the Upgrade of the Cluster as outlined above before creating any Disk groups on the newly added host(s).
If you are adding or replacing Disks in the midst of an Upgrade, ensure that they are formatted with the appropriate legacy on-disk format version, if applicable.
For more information, see How to format vSAN Disk Groups with a legacy format version (326580) and Failure to promote CMMDs version resulting in vSAN cluster to become partitioned during upgrade (326888). Certain vSAN behavior changes are controlled by the on-disk format present. It is important that newer on-disk format versions are not introduced into a mixed-version Cluster. For more information, see Understanding vSAN on-disk format versions and compatibility (327034)
Failure to adhere to these best practices may result in one or more of the following issues:
- Unexpected Network partitions
- Unexpected loss of Data availability
- vSAN Cluster instability
Additional Information
Attachments
Feedback
Yes
No
Powered by
