ESXi hosts disconnect and Active Directory authentication fails after vCenter Server upgrade

Products

• VMware vCenter Server

Issue/Introduction

• After you upgrade vCenter Server to version 8.0U3g (build 24853646), all ESXi hosts display as disconnected in the vSphere Client and you cannot authenticate using Active Directory credentials.
• The local [email protected] account continues to function, but you cannot manage ESXi hosts or access vCenter using domain accounts.
• This issue persists even after reverting vCenter to a pre-upgrade snapshot.
• The vCenter Server services may start successfully, but the management plane remains non-functional.

Additional symptoms reported:

• Lost AD access to vCenter after patching
• All hosts got disconnected following the upgrade
• Reverting to snapshot does not resolve the issue

Environment

vCenter Server 8.0.x

Cause

This issue occurs due to two distinct but related problems that can happen during vCenter Server upgrades:

Expired vCenter Certificates: During the upgrade process, vCenter Server validates SSL certificates for secure communication channels. When certificates in the certificate store (including machine SSL certificates, Security Token Service certificates, or Single Sign-On certificates) have expired before or during the upgrade, the validation fails. ESXi hosts require valid certificates to establish trusted connections with vCenter Server. Without these valid certificates, the hosts cannot authenticate and appear as disconnected. Similarly, Active Directory authentication relies on valid certificates for LDAPS (LDAP over SSL) connections to domain controllers. Expired certificates break this trust chain, preventing any domain account from authenticating to vCenter.

Missing DNS Configuration After Snapshot Revert: When you revert vCenter Server to a previous snapshot, the network configuration returns to the state captured in that snapshot. If DNS server addresses were not configured at the time the snapshot was created, or if they were added after the snapshot was taken, the revert removes this DNS configuration. Without DNS resolution, vCenter cannot resolve the fully qualified domain names (FQDNs) of Active Directory domain controllers or ESXi hosts. This prevents vCenter from communicating with these systems even if certificate issues are resolved.

These two conditions often occur together because administrators may revert to a snapshot in an attempt to recover from the upgrade issue, not realizing that the snapshot revert introduces an additional DNS configuration problem. Both issues must be resolved for vCenter to regain full functionality.

Resolution

Step 1: Diagnose the Issue

1. Run the vSphere Diagnostic Tool (VDT) to identify certificate and DNS issues:

   • Follow Using the VCF Diagnostic Tool for vSphere (VDT) to download and run the tool
   • Review the VDT report output for certificate expiration warnings and DNS resolution failures

2. Alternatively, check certificates manually using Certificate Manager:

   • Log in to VAMI at https://<vCenter_FQDN>:5480
   • Navigate to Certificate Manager
   • Review certificate expiration dates for Machine SSL, STS Signing, and SSO certificates

3. Verify DNS configuration:

   • In VAMI, navigate to Networking
   • Check if DNS server addresses are configured
   • Test DNS resolution from SSH: nslookup <domain_controller_FQDN>

Step 2: Determine Certificate Scope (if certificates are expired)

Identify which certificates are expired and whether you are using VMCA self-signed (default) or custom CA-signed certificates. This determines which vCert menu options to use.

Step 3: Resolve Certificate Issues Using vCert

1. Download and run the vCert tool following vCert - Scripted vCenter expired certificate replacement

2. Use the tool's certificate health check to identify specific certificate issues

3. Follow the KB guidance to select the appropriate certificate replacement options based on your findings

4. After certificate replacement, restart all vCenter services and verify they start successfully

Step 4: Resolve DNS Configuration Issues

1. Log in to VAMI at https://<vCenter_FQDN>:5480

2. Navigate to Networking → Edit Settings

3. Add primary and secondary DNS server IP addresses and save

4. Verify DNS resolution works from SSH

For detailed steps, see Update DNS Server IP address for vCenter Server

Step 5: Verify and Reconfigure Active Directory Integration

For Integrated Windows Authentication (IWA):

• Navigate to Administration → Single Sign-On → Configuration → Identity Providers
• If the Windows Session Authentication identity source shows errors, remove and re-add it

For Active Directory over LDAP:

• If the Active Directory identity source shows errors, remove and re-add it
• Use LDAPS (recommended best practice for vSphere 8.x as IWA is being deprecated)
• Follow Configuring a vCenter Single Sign-On Identity Source using LDAP with SSL (LDAPS)

Test Active Directory authentication with a domain account after reconfiguration.

Step 6: Reconnect ESXi Hosts

After certificate replacement and DNS resolution, ESXi hosts may need manual reconnection:

1. For each disconnected host, right-click and select Connect

2. If connection fails with certificate errors:

   • Right-click the host and select Reset Certificate
   • Enter root credentials for the ESXi host when prompted
   • Accept the new certificate thumbprint

3. If a host was reverted from a snapshot and vCenter doesn't recognize its certificate:

   • Remove the host from inventory
   • Re-add the host and provide FQDN, root credentials, and accept the certificate thumbprint

Step 7: Verify Resolution

1. Confirm all ESXi hosts show "Connected" status

2. Verify Active Directory authentication works

3. Check vCenter Server health in VAMI to confirm all indicators show green and certificate warnings are resolved