vSAN Stretch Clusters - How to Replace the Witness Appliance/Host

Products

• VMware vSAN

Issue/Introduction

Steps for replacing the witness appliance/host for vSAN Stretch Clusters, 2-Node Stretch/Robo Clusters, 2-Node Stretch Direct Connect Clusters.

Environment

VMware vSAN (All Versions)

Resolution

There are two ways of replacing the witness appliance/host either a witness using a new name and IP address or reusing the old name and IP

Deploying a new witness appliance with a different name and IP

1. Download the witness appliance OVA with the same ESXi Build version that matches the physical hosts in the cluster.
   Note: If the exact build version is not available for download, then download an ESXi build version lower than the physical ESXi build version so you can then upgrade the witness to match the rest of the cluster. 
   
   
2. Deploy the new vSAN witness appliance to a datastore of a physical ESXi host or cluster.  
   Note: This host/cluster can not be part of the vSAN cluster as the witness host provides quorum and if deployed to the vSAN cluster this creates a circular redundancy and is not supported.
   
   
3. Add the witness host to vCenter
   
   
4. If the exact version is not available for download and you downloaded the closest version but not higher than the rest of the cluster upgrade the witness appliance to match the rest of the cluster.
   

   Note: If the vSAN on-disk format version has not been upgraded to the latest version and a Skyline Health warning for "Disk format version" is present, configuring a stretched cluster with a newly deployed Witness Appliance can lead to a disk format version mismatch.

   In these scenarios, you must align the Witness Appliance with the existing data nodes before completing the configuration:

   1. Modify the virsto.DiskFormatVersion on the new Witness Appliance to the legacy version that matches your data nodes.

   2. Follow the steps outlined in the KB article: "How to format vSAN Disk Groups with a legacy format version."

   3. Once the versions match, proceed with the stretched cluster configuration.

5. Select the vSAN Cluster > Configure > vSAN > Fault Domains > Select Change Witness Host and follow the wizard to change the witness appliance 

Deploying a new witness appliance with the same name and IP

1. Download the witness appliance OVA with the same ESXi Build version that matches the physical hosts in the cluster.
   Note: If the exact build version is not available for download, then download an ESXi build version lower than the physical ESXi build version so you can then upgrade the witness to match the rest of the cluster. 
   
   
2. Document the config info of the current witness appliance if possible
   
   
3. Ensure all vSAN objects are healthy or in a reduced availability state due to missing witness components 
   
   
4. If all objects are healthy Select the vSAN Cluster > Configure > vSAN > Fault Domains > Select Disable Stretch cluster
   Note: vSAN Objects will go into a reduced availability state if not already
   
   
5. Power down the old witness appliance and remove it from vCenter inventory
   
   
6. Deploy the new vSAN witness appliance to a datastore of a physical ESXi host or cluster.  
   Note: This host/cluster can not be part of the vSAN cluster as the witness host provides quorum and if deployed to the vSAN cluster this creates a circular redundancy and is not supported.
   
   
7. If the exact version is not available for download and you downloaded the closest version but not higher than the rest of the cluster upgrade the witness appliance to match the rest of the cluster.
   

   Note: If the vSAN on-disk format version has not been upgraded to the latest version and a Skyline Health warning for "Disk format version" is present, configuring a stretched cluster with a newly deployed Witness Appliance can lead to a disk format version mismatch.

   In these scenarios, you must align the Witness Appliance with the existing data nodes before completing the configuration:

   1. Modify the virsto.DiskFormatVersion on the new Witness Appliance to the legacy version that matches your data nodes.

   2. Follow the steps outlined in the KB article: "How to format vSAN Disk Groups with a legacy format version."

   3. Once the versions match, proceed with the stretched cluster configuration.

8. Select the vSAN Cluster > Configure > vSAN > Fault Domains > Select Configure Stretch cluster and follow the wizard to configure the stretch cluster 

If assistance is required with replacing the witness open a case with vSAN support.

Note: If the vSAN witness host is unavailable or in a failed state, the objects associated with that witness will remain in a Reduced Availability state. Adding or replacing the witness appliance does not cause data loss or additional impact; it only restores full availability and compliance once the new witness is successfully configured.

When deploying a replacement witness, ensure that the network configuration, ESXi build/version, and on-disk format version of the new witness appliance are consistent with the existing data nodes and the previous witness host.

Additional Information

Configuring Stretched Cluster Results in "A specified parameter was not correct - Witness host already configured in stretched cluster"