After restoring vCenter Server, Auto Deploy rule creations display Error "Invalid Fault Cause: Cannot complete login due to an incorrect user name or password"

Products

• VMware vCenter Server

Issue/Introduction

• After restoring vCenter Server from a backup, you are unable to create deploy rules for Auto Deploy

• The operation returns an error such as:

          Invalid Fault Cause: Cannot complete login due to an incorrect user name or password

• vSphere Client logs show a message similar to:

         /var/log/vmware/vsphere-ui/logs/vsphere_client_virgo.log

    [YYYY-MM-DDTHH:MM:SS] [ERROR] data-service-pool-795  70001335 100007 200002 com.vmware.vise.util.XMLUtil Error when loading xml string java.lang.NullPointerException: nu
    at java.io.StringReader.<init>(StringReader.java:50)
    at com.vmware.vise.util.XMLUtil.parseXml(XMLUtil.java:236)
    at com.vmware.vsphere.client.vcaddons.impl.AutoDeployPropertyProviderImpl.parseAutoDeployConfiguration(AutoDeployPropertyProviderImpl.java:200)
    at com.vmware.vsphere.client.vcaddons.impl.AutoDeployPropertyProviderImpl.getAutoDeployConfiguration(AutoDeployPropertyProviderImpl.java:147)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

Environment

• vCenter 7.x
• vCenter 8.x

Cause

The issue is caused since the affected service communicates with the vpxd service and post the database restore the certificate and key was from the previous instance. The entries from previous instance is restored with the database under table vpx_ext.

Resolution

To resolve the issue, follow the below steps.

1. Log in to the vCenter Server via shell or SSH
2. Create a temporary directory under root
   mkdir /certificate
3. Create a copy of the certificate and key from the vpxd-extension store

1. /usr/lib/vmware-vmafd/bin/vecs-cli entry getcert --store vpxd-extension --alias vpxd-extension --output /certificate/vpxd-extension.crt
2. /usr/lib/vmware-vmafd/bin/vecs-cli entry getkey --store vpxd-extension --alias vpxd-extension --output /certificate/vpxd-extension.key

    4. Update the service endpoint using the vpxd-extension certificate

1. python /usr/lib/vmware-vpx/scripts/updateExtensionCertInVC.py -e com.vmware.vim.eam -c /certificate/vpxd-extension.crt -k /certificate/vpxd-extension.key -s <FQDN> -u Administrator@<SSO Domain> -p <SSO Password>
2. python /usr/lib/vmware-vpx/scripts/updateExtensionCertInVC.py -e com.vmware.rbd -c /certificate/vpxd-extension.crt -k /certificate/vpxd-extension.key -s <FQDN> -u Administrator@<SSO Domain> -p <SSO Password>
3. python /usr/lib/vmware-vpx/scripts/updateExtensionCertInVC.py -e com.vmware.imagebuilder -c /certificate/vpxd-extension.crt -k /certificate/vpxd-extension.key -s <FQDN> -u Administrator@<SSO Domain> -p <SSO Password>

     Note: If the above commands do not work, try the commands without -p <SSO Password>. A prompt for the password will appear instead. 
   python /usr/lib/vmware-vpx/scripts/updateExtensionCertInVC.py -e com.vmware.vim.eam -c /certificate/vpxd-extension.crt -k /certificate/vpxd-extension.key -s <FQDN> -u Administrator@<SSO Domain>
   python /usr/lib/vmware-vpx/scripts/updateExtensionCertInVC.py -e com.vmware.rbd -c /certificate/vpxd-extension.crt -k /certificate/vpxd-extension.key -s <FQDN> -u Administrator@<SSO Domain>
   python /usr/lib/vmware-vpx/scripts/updateExtensionCertInVC.py -e com.vmware.imagebuilder -c /certificate/vpxd-extension.crt -k /certificate/vpxd-extension.key -s <FQDN> -u Administrator@<SSO Domain> 
   

     5. Restart the services
        service-control --restart vmware-eam
    service-control --restart vmware-imagebuilder
    service-control --restart vmware-rbd-watchdog