SDDC Manager add cluster validation error - Failed to get upgrade manager status. Provided domain ID is invalid

Products

• VMware SDDC Manager

Issue/Introduction

An issue was encountered during the creation of a new cluster within SDDC Manager. Specifically, the process failed with three intended hosts remaining in an unassigned state, preventing the successful deployment of the cluster.

Verification of the issue confirmed that the SDDC Manager deployment process terminated with the following error message: "The minimum requirement of three clean, unassigned hosts has not been met. Hosts must be commissioned with at least two physical NICs and the ESXi version must be compatible with the lowest ESXi version present in the domain:

ERROR:

Failed to get upgrade manager status. Provided domain ID is invalid

The minimum requirement of three clean, unassigned hosts has not been met. Hosts must be commissioned with
at least two physical NICs and the ESXi version must be compatible with the lowest ESXi version present in the
domain.

Failed to fetch base esx version.

Environment

VMware Cloud Foundation 5.x

Cause

Check Host Attributes by the running the command: psql -h localhost -U postgres -d platform -c "\x" -c "select id,hostname,host_attributes from host":

Volume Name	VMFS UUID	Extent Number	Device Name	Partition
####-T1-H01	687fb461-######-3025-####cc001f4	0	naa.614####0000001d	1
datastore1	68811086-######-aca3-####cc001ca	0	naa.600####409f44	8
OSDATA	68811086-######-9836-####cc001ca	0	naa.600####409f44	7
datastore1	68681b26-######-784d-####cc001f2	0	naa.600####652415	8
####-T1-H01	688a5416-######-28da-####cc001f2	0	naa.614####0000001d	2
OSDATA	68681b26-######-516b-####cc001f2	0	naa.600####652415	7

 

The cluster creation failure in SDDC Manager, with hosts remaining unassigned, was caused by a fundamental inconsistency in LUN presentation across the three ESXi hosts. Specifically, the intended shared storage volumes (e.g., datastore1, OSDATA) exhibited UUID mismatches across the hosts, as evidenced by psql host attribute queries. This misalignment prevented SDDC Manager from identifying the hosts as suitable for cluster integration. Resolution requires removing existing LUNs and reconfiguring storage from scratch on all affected hosts to ensure consistent LUN identification and shared access.

The cluster creation failure in SDDC Manager, resulting in unassigned hosts, was directly attributable to inconsistent LUN configurations.

This data demonstrates that volumes such as datastore1 and OSDATA are presented with varying UUIDs across different host entries, indicating a fundamental mismatch in how shared storage was configured and perceived. This inconsistency prevented SDDC Manager from successfully commissioning the hosts and forming the cluster.

To resolve this, all existing LUNs must be removed, and a comprehensive storage reconfiguration, including provisioning new LUNs, is required on all three hosts to ensure uniform and consistent storage presentation.

Resolution

The resolution implemented to address the storage and host configuration discrepancies involved a series of strategic steps:

1. Shared Datastore Provisioning: A new shared datastore was manually established leveraging Fibre Channel over Ethernet (FCoE) connectivity.
2. Host Decommissioning: All previously affected ESXi hosts were systematically decommissioned from their existing configuration states within the environment.
3. Host Re-commissioning: Following decommissioning, these hosts were subsequently re-commissioned into the SDDC Manager system.
4. Cluster Integration: Upon successful re-commissioning, the hosts were seamlessly integrated into the newly provisioned cluster.