vCenter Server and SDDC Manager Offline Following vSAN networking event - VMware Cloud Foundation

Products

• VMware vSAN

Issue/Introduction

You may experience a management plane outage where vCenter Server and SDDC Manager are unreachable following a network disruption. This occurs when vSphere High Availability (HA) triggers virtual machine failovers while backend vSAN storage or management networking is unstable.

• vCenter Server powers on but fails to acquire an IP address.
• SDDC Manager remains offline or unreachable.
• Direct ESXi host access is blocked due to unavailable root credentials (if managed by SDDC Manager).
• vSphere Client shows invalid or stale guest virtual machine registrations.

Environment

• VMware ESXi 8.0.3 (all versions)

• VMware vCenter Server 8.0.3 (all versions)

• VMware SDDC Manager 5.2.3

Cause

A physical network outage disrupts backend vSAN storage, triggering vSphere HA migrations. When the management VMs (vCenter and SDDC Manager) attempt to restart, the transient network instability prevents the virtual network adapters from properly initializing or acquiring an IP address from the management network.

If SDDC manager was impacted by the outage and did not boot correctly, root credentials will not be accessible via SDDC until the Appliance has been powered and and is accessible. 

Resolution

1. Locate your valid ESXi root credentials to restore direct administrative access to the physical hosts.
2. Ensure that the physical network connectivity and vSAN storage health are fully restored.
3. Log in to the ESXi host management interface or DCUI.
4. Power on/reboot the vCenter Server and SDDC Manager virtual machines. This allows the virtual network interfaces to re-initialize and acquire their assigned IP addresses.
5. Verify that vCenter Server successfully binds to its IP address and that all management services initialize.
6. Clear any stale or invalid guest virtual machine registrations that appear in vCenter inventory.
7. Confirm that SDDC Manager service health is restored to "Operational."
   
   
   
   

Additional Information

If the Root password for the ESXI hosts is not available you can recovery via the following methods. 

• • Methods to recover the ESXi host root password

  • Rebuilding a vSAN cluster after reinstalling ESXi on all hosts without vCenter while preserving production data