Symptoms:
"VC/PSC password is expired for "VCName". SSH to "VCName",switch to bash shell using command 'shell' and run the command 'passwd'.Reset the password to the same as earlier. Please retry the upgrade once the upgrade is available again"
/usr/lib/vmware-vmafd/bin/dir-cli user find-by-name --account administrator --level 2
/var/log/vmware/vcf/operationsmanager/operationsmanager.log shows YYYY-MM-DDTHH:MM:SS ERROR [################,####] [c.v.v.p.s.PasswordValidationService,om-exec-15] Password expiry retrieval is failed for entity: PSC, credential type: SSO, address: "fqdn"com.vmware.vim.sso.admin.exception.InternalError: pwdLastSet(86399999913600) should be less or equal to currentTimeSec(1646425827) at com.vmware.vim.sso.admin.client.vmomi.impl.VmomiClientCommand.execute(VmomiClientCommand.java:172) at com.vmware.vim.sso.admin.client.vmomi.impl.VmomiClientCommand.executeEnsuringDomainErrorIs(VmomiClientCommand.java:220) at com.vmware.vim.sso.admin.client.vmomi.impl.VmomiClientCommand.executeEnsuringDomainErrorIs(VmomiClientCommand.java:202) at com.vmware.vim.sso.admin.client.vmomi.impl.PrincipalSelfManagementImpl.getDaysRemainingUntilPasswordExpiration(PrincipalSelfManagementImpl.java:128) at com.vmware.vcf.passwordmanager.update.changers.SSOChanger.getPasswordExpiry(SSOChanger.java:200) at com.vmware.vcf.passwordmanager.service.PasswordValidationService.testPasswordExpiry(PasswordValidationService.java:435) at com.vmware.vcf.passwordmanager.service.PasswordValidationService.validatePasswordForEntity(PasswordValidationService.java:273) at com.vmware.vcf.passwordmanager.validation.utils.CredentialsValidationTaskExecutor$2.call(CredentialsValidationTaskExecutor.java:141) at com.vmware.vcf.passwordmanager.validation.utils.CredentialsValidationTaskExecutor$2.call(CredentialsValidationTaskExecutor.java:136) at org.springframework.cloud.sleuth.instrument.async.TraceCallable.call(TraceCallable.java:70) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)YYYY-MM-DDTHH:MM:SS WARN [################,####] [c.v.v.v.c.h.i.HttpConfigurationCompilerBase$ConnectionMonitorThreadBase,om-exec-10] Shutting down the connection monitor.YYYY-MM-DDTHH:MM:SS INFO [################,####] [c.v.v.s.a.c.v.impl.AbstractClient,om-exec-10] Client was disposed successfullyYYYY-MM-DDTHH:MM:SS ERROR [################,####] [c.v.v.p.s.PasswordValidationService,om-exec-10] Password expiry retrieval is failed for entity: PSC, credential type: SSO, address: "fqdn"com.vmware.vim.sso.admin.exception.InternalError: pwdLastSet(86399999913600) should be less or equal to currentTimeSec(1646425827) at com.vmware.vim.sso.admin.client.vmomi.impl.VmomiClientCommand.execute(VmomiClientCommand.java:172) at com.vmware.vim.sso.admin.client.vmomi.impl.VmomiClientCommand.executeEnsuringDomainErrorIs(VmomiClientCommand.java:220) at com.vmware.vim.sso.admin.client.vmomi.impl.VmomiClientCommand.executeEnsuringDomainErrorIs(VmomiClientCommand.java:202) at com.vmware.vim.sso.admin.client.vmomi.impl.PrincipalSelfManagementImpl.getDaysRemainingUntilPasswordExpiration(PrincipalSelfManagementImpl.java:128) at com.vmware.vcf.passwordmanager.update.changers.SSOChanger.getPasswordExpiry(SSOChanger.java:200) at com.vmware.vcf.passwordmanager.service.PasswordValidationService.testPasswordExpiry(PasswordValidationService.java:435) at com.vmware.vcf.passwordmanager.service.PasswordValidationService.validatePasswordForEntity(PasswordValidationService.java:273) at com.vmware.vcf.passwordmanager.validation.utils.CredentialsValidationTaskExecutor$2.call(CredentialsValidationTaskExecutor.java:141) at com.vmware.vcf.passwordmanager.validation.utils.CredentialsValidationTaskExecutor$2.call(CredentialsValidationTaskExecutor.java:136) at org.springframework.cloud.sleuth.instrument.async.TraceCallable.call(TraceCallable.java:70) at java.util.concurrent.FutureTask.run(FutureTask.java:266) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at java.lang.Thread.run(Thread.java:748)
Note: The preceding log excerpts are only examples. Date, time and environmental variables may vary depending on your environment.
VMware Cloud Foundation 4.x
The error message from vSphere SDK incorrectly names pwdLifetime as pwdLastSet. pwdLastSet(86399999913600) should be less or equal to currentTimeSec(1646425827). And the epochs 86399999913600 corresponds to around 980931 days.
This issue is resolved in VMware Cloud Foundation version 4.5.
Workaround:
To workaround this issue set the password policy to any number less than or equal to 9999.
Perform the following steps to workaround this issue.
VC/PSC node./usr/lib/vmware-vmafd/bin/dir-cli user find-by-name --account administrator --level 2 (Verify the number of days)/usr/lib/vmware-vmafd/bin/dir-cli user modify --account administrator --password-expires (Update the expiration policy)NOTE: This is a corner case. Exists only when the Maximum lifetime password policy for vCenter local accounts is set to a number greater than 9999.
When the password of SSO account is set to be expired and the maximum lifetime days in local password policy is set to 999999999. (Allowed as per vSphere docs: Edit the vCenter Single Sign-On Password Policy