Failed to upgrade VCF management lifecycle from 9.1.0 to 9.1.1 unable to create vSphere client: ServerFaultCode: Cannot complete login due to an incorrect user name or password.] [VCFMS-HEALTH-002]

Products

• VCF Operations

Issue/Introduction

When attempting to upgrade the VCF Fleet Lifecycle from version 9.1 to 9.1.1, the upgrade task fails at the "Component Stage Prechecks" step with the following error:

Error Platform Health Check Error [platform-vmsp-platform-validate-resources-vsphere-resources : unable to create vSphere client: ServerFaultCode: Cannot complete login due to an incorrect user name or password.] [VCFMS-HEALTH-002]

On vCenter Server, check log file /var/log/vmware/sso/vmware-identity-sts.log , found error message:
  YYYY-MM-DDThh:mm:ss.###Z INFO vmidentity-sts ## [vc@### threadName="tomcat-http--##" logger="com.vmware.identity.sts.ws.SOAPFaultHandler" corId="#########-####-####-####-########"] Returning a SOAP Fault with code: ns0:FailedAuthentication and description: Password of the user logging on is expired. :: Password of the user logging on is expired.
:: User account expired: {Name: svc-vcfsp-vc-#####, Domain: vsphere.local}

Environment

VCF 9.1.1

Cause

The service accounts utilized by the VCF Services Runtime to authenticate with vCenter have expired.

This can be confirmed by checking  the log file:   /var/log/vmware/sso/vmware-identity-sts.log

  cat  |    /var/log/vmware/sso/vmware-identity-sts.log  | grep -i  "Password of the user logging on is expired"  | grep svc-vcfsp-vc

Resolution

To resolve this issue, you must run the script service-account-remediation.sh in KB#448804 

 

a)Verify that the output of script are all 'successfully‘.

b) Check /var/log/vmware/sso/svcaccountmgmt.log, ensure  the following log message can be found: 

    Password Reset successful for accountUPN :svc-vcfsp-vc-###@##@VSPHERE.LOCAL

 

After confirming the script has been executed successfully, then retry the task in VCF  Operations.