SDDC Manager Patching Fails Due to SSH Authentication Errors with vCenter Server in VCF 9.x

Products

• VMware vSphere ESXi

Issue/Introduction

During VMware Cloud Foundation (VCF) lifecycle management operations, such as patching or component upgrades, SDDC Manager must establish SSH connectivity to vCenter Server components. If the root password stored in the SDDC Manager vault does not match the actual root password of the vCenter Server, operations will fail. This article outlines how to identify and resolve these authentication mismatches.

• SDDC Manager patching or upgrade workflows fail intermittently or consistently during pre-check or execution phases.
• The SDDC Manager UI becomes unresponsive or inaccessible for specific lifecycle management tasks.
• Log file /var/log/vmware/vcf/commonsvcs/vcf-commonsvcs.log contain VcfSshException: Authentication failed for host <vCenter_FQDN> messages.
• Example log error:  ERROR [common, e9d62427b34c4415, 7d18] [c.v.e.s.c.s.a.r.CommonServicesSamlMetadataExchangeController] Exception in shellAgnosticCommandExec for host ####.####.#### com.vmware.vcf.secure.ssh.errors.VcfSshException: Authentication failed for host ####.####.####
• Inability to SSH manually into vCenter Server using the credential expected by the SDDC Manager.

Environment

VMware Cloud Foundation (VCF) 9.x

Cause

The failure is caused by a credential mismatch between the root password stored in the SDDC Manager's internal credential vault and the actual root password set on the target vCenter Server.

Resolution

Follow the steps below to synchronize the credentials and resume VCF operations.

1. Identify the correct vCenter root password stored in the SDDC Manager vault by using the lookup_passwords utility via the SDDC Manager shell.
2. SSH into the vCenter Server as root using the current known password.
3. Clear the password history to allow reusing the credential found in step 1: echo "" > /etc/security/opasswd
4. Reset the root password on the vCenter Server to match the value identified in step 1: passwd root
5. If you initiated a patching workflow that failed, revert the SDDC Manager snapshot.
6. Restart the patching process through the Plan Updates workflow.

Additional Information

If the issue persists, ensure that all recommended pre-upgrade health checks are completed. To speak with a customer representative or a Support Engineer see Contact Support. Scroll to the bottom of the page and click on your respective region.