Error: VSAN Cluster validation failed in SDDC Manager due to network pool IP or MTU conflicts

Products

• VMware SDDC Manager / VCF Installer

Issue/Introduction

• Workload Domain deployment in SDDC Manager fails during the disk group creation or vSAN cluster validation stage.
• vMotion connectivity with some other hosts and vMotion: vMotion MTU check (ping with large packet size).
• vSAN Health Check reports failures for vMotion: vMotion connectivity with some other hosts and vMotion: vMotion MTU check (ping with large packet size).
• vSAN host frequently becomes Network partitioned, then after sometime the host joins back without any intervention.
• Skyline Health reports "vSAN cluster partition" or "vSAN health degraded".
• SDDC Manager: /var/log/vmware/vcf/operationsmanager/domainmanager/domainmanager.log
IP address conflicts or MTU mismatches
Error Message: VSAN Cluster validation failed on ####
• ESX: /var/run/log/vobd.log
[vob.net.vmknic.ip.duplicate] A duplicate IP address was detected for ####.####.####.#### on interface vmk# . The current owner is ####:####:####:####:####:####.

Environment

• VCF 9.x
• VCF 5.2.x

Cause

This failure occurs when host vmkernel adapters (typically used for vSAN or vMotion) have IP addresses that are already in use on the network or fall within a range already managed by a different SDDC Manager network pool. Connectivity and MTU alarms are triggered when vMotion or vSAN traffic is interrupted by these IP conflicts or incorrect MTU settings (e.g., MTU not set to 1500).

Resolution

If this is impacting the management domain reach out to Broadcom Support.
 
If this is impacting a workload domain, Redeploy the workload domain and assign a new, non-conflicting network pool. Follow these steps:

Note : Take a Snapshot and/or Backup of SDDC Manager before making any changes.

1. Delete the existing Workload Domain by following the procedure outlined in Delete a VI Workload Domain.
2. Remove the associated Network Pool from the SDDC Manager UI by following Delete a Network Pool.
3. Decommission the ESXi hosts that were part of the Workload Domain by following Decommission Hosts.
4. Create a new Network Pool in SDDC Manager by following Create a Network Pool.
5. Update the physical network routing and configuration, and then redeploy the ESXi hosts to ensure connectivity with the new IP address range.
6. Commission the ESXi hosts back into SDDC Manager by following Commission Hosts.
7. Create a new Workload Domain using the newly commissioned hosts by following the procedure in Deploy a VI Workload Domain Using the SDDC Manager UI.

If deleting the existing Workload Domain is not feasible in a Production Environment, reach out to Broadcom Support.

Additional Information

Network Pool modification for existing domains in SDDC Manager