Avoid taking a snapshot of the VMware vSAN Witness Appliance

Products

VMware vSAN VMware vSAN 6.x VMware vSAN 7.x VMware vSAN 8.x

Issue/Introduction

Taking snapshots of the vSAN Witness Appliance is considered bad practice and is not supported by VMware. Snapshots can introduce these types of issues that impacts the stability of a vSAN cluster:

Latency
Metadata consistency disruption
Split-brain scenarios

This article explains the risks associated with snapshotting the vSAN Witness Appliance and provides alternative approaches for backup and recovery.

Environment

VMware vSAN 6,x, 7.x, 8.x environment.

Cause

Why Taking Snapshots of the vSAN Witness Appliance is Not Supported

1. Performance and Latency Issues

Snapshots introduce additional disk I/O overhead, which can slow down witness traffic and impact performance.

2. Risk of Split-Brain Conditions

If a snapshot is restored, the vSAN Witness may contain outdated metadata, leading to a situation where different hosts disagree on object ownership.
This can cause data inconsistencies and force unnecessary rebuilds.

3. Complications During Failover

A restored Witness snapshot may be out of sync with the active vSAN nodes in the cluster.
This may cause metadata inconsistencies, increasing the risk of a degraded object accessible state (inaccessible, reduced availability).

4. VMware Official Guidance

VMware does not support the creation of snapshots for a backup of the vSAN Witness Appliance. For more information, see:

Resolution

Recommended Alternatives

1. Deploy a New vSAN Witness Appliance

If the vSAN Witness Appliance fails, VMware recommends to redeploying a fresh Witness Appliance and reconfiguring it.
This ensures metadata consistency and avoids complications from outdated snapshots.
In the future, instead of having backups, clones or snapshots of the vSAN Witness Appliance, replace or change the witness host for a vSAN stretched cluster. For more information, see Change the Witness Host.

2. Monitor vSAN Witness Health

Regularly check vSAN Skyline Health in vSphere:
- Navigate using the vSphere Client > Cluster > Monitor > vSAN > Skyline Health.
- Ensure the Witness node is in a healthy state.
- For more information, see vSAN Monitoring and Troubleshooting.

3. Perform Proper Maintenance Before Updates

If for any reason you must take a snapshot, ensure that:
- The vSAN Witness Appliance is powered off.
- The snapshot is deleted immediately after the scheduled maintenance.
- It is only used in test environments and not production.

Conclusion

Taking snapshots of the vSAN Witness Appliance is a high-risk action that can lead to latency issues, split-brain conditions, and vSAN metadata corruption. VMware strongly advises NOT using snapshots and instead recommends alternative backup and recovery methods, such as using external backups and redeploying the vSAN Witness Appliance when necessary.

For more information, see the VMware vSAN Documentation.