Registering VASA provider fails with error "A problem was encountered while registering the provider"
search cancel

Registering VASA provider fails with error "A problem was encountered while registering the provider"

book

Article ID: 368313

calendar_today

Updated On:

Products

VMware vCenter Server 8.0

Issue/Introduction

  • When trying to register VASA provider on vCenter, the following error appears:  "A problem was encountered while registering the provider.”

  • vCenter has been upgraded to 8.x from earlier versions eg.,5.x
  • The following error snippets are observed in the /var/log/vmware/vmware-sps/sps.log Log file:

virtualHostInfo = (VirtualHostInfo) {
SNI = null
multiVcSupported = true
port = null
serviceHost = null
vcGuid = XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX       <----- This entry of the vcGuid will be in uppercase
}
}

YYYY-MM-DDTHH:MM:SS [pool-38-thread-4] INFO  opId=mamo8ouk-4614-auto-3ko-h5:70000458 com.vmware.vim.storage.common.security.CommonActivationValidator - [getRolesFromRoleCacheObject] Validating session for user <user-name> for method QuerySmsTaskInfo having correlator 3647
YYYY-MM-DDTHH:MM:SS [pool-29-thread-3] ERROR opId=mamo8ouk-4614-auto-3k7-h5:70000458 com.vmware.vim.sms.client.VasaClientImpl - [removeVirtualHost] Remote exception
org.apache.axis2.AxisFault: Retrieved vasa version is null. Hostname verification could not be done.

YYYY-MM-DDTHH:MM:SS [pool-3-thread-8] INFO  opId=mamo8ouk-4614-auto-3k7-h5:70000458 com.vmware.cis.localkeyvalue.client.impl.KVClientImpl - KV Client login by SamlToken successful
YYYY-MM-DDTHH:MM:SS [pool-3-thread-8] INFO  opId=mamo8ouk-4614-auto-3k7-h5:70000458 com.vmware.vim.storage.common.serviceclient.SamlTokenLoginHelper - Login successful for KV client.
YYYY-MM-DDTHH:MM:SS [pool-29-thread-3] ERROR opId=mamo8ouk-4614-auto-3k7-h5:70000458 com.vmware.vim.sms.kvstore.KvStorePersistenceManager - VasaProviderInfo could not be found for the provider: ########-####-####-####-############
YYYY-MM-DDTHH:MM:SS [pool-29-thread-3] ERROR opId=mamo8ouk-4614-auto-3k7-h5:70000458 com.vmware.vim.sms.provider.vasa.VasaProviderInfoPersistenceManager - Exception occured when removing the provider information in KV store:########-####-####-####-############
com.vmware.vim.sms.fault.KvNotFoundException: VasaProviderInfo could not be found for the provider: 7e16a896-fdec-460a-93ef-f92f00dd3c6b
YYYY-MM-DDTHH:MM:SS [pool-29-thread-3] WARN  opId=mamo8ouk-4614-auto-3k7-h5:70000458 com.vmware.vim.sms.provider.vasa.VasaProviderInfoPersistenceManager - [cleanProvider] Failed to remove provider from KV store!
com.vmware.vim.sms.fault.DBPersistenceException: Exception occured when removing the provider information in KV store########-####-####-####-############

YYYY-MM-DDTHH:MM:SS [pool-29-thread-3] ERROR opId=mamo8ouk-4614-auto-3k7-h5:70000458 com.vmware.vim.storage.common.VmodlErrorStrings - [init] Provider creation failed
java.lang.NullPointerException: null
YYYY-MM-DDTHH:MM:SS [pool-29-thread-3] INFO  opId=mamo8ouk-4614-auto-3k7-h5:70000458 com.vmware.vim.sms.provider.ProviderCache - [removeSNIEntry] Removing SNI entry for URL https://<VASA_provider_URL>:<port>/version.xml

  • When we execute wget for the .XML file highlighted in the sps logs above, we notice that the same vcGuid, highlighted in the above snippets, is in lower case.

    Command:
    wget <URL>.xml --no-check-certificate

    Output:

    <?xml version="1.0" encoding="UTF-8" standalone="yes"?><vasa-provider><supported-versions><version id="4" serviceLocation="/vasa"/><version id="5" serviceLocation="/vasa"/><version id="6" serviceLocation="/vasa"/><version id="7" serviceLocation="/vasa"/></supported-versions><serviceVirtualHost><virtualHost vcguid="########-####-####-####-############" virtualHostName="vasa-vhost-########-####-####-####-############" serviceHost="##.##.##.###"/><virtualHost vcguid="xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx" virtualHostName="vasa-vhost-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx.purestorage.com" serviceHost="##.##.##.###"/></serviceVirtualHost></vasa-provider>


     

Cause

This behavior is typically observed in vCenter Server instances that have been upgraded from version 5.x.
During such upgrades, the files /etc/vmware-vpx/instance.cfg and /etc/vmware-vpx/firstboot/vpxd-service-spec.prop retained  vcGuid in uppercase, as seen in the /var/log/vmware/vmware-sps/sps.log.
However, the version.xml file contains the vcGuid in lowercase. This case-sensitivity discrepancy is the root cause of the registration failure.

Resolution

  1. Update "instance.cfg" and "vpxd-service-spec.prop" to use lowercase letters for the vCenter GUID. 
       
       a.Take a snapshot of vCenter
       b.Open SSH connection to vCenter as root
       c.Edit "/etc/vmware-vpx/instance.cfg" and update the instanceUuid value from uppercase letters to lowercase letters.
       
        # vi /etc/vmware-vpx/instance.cfg
        Change line "instanceUuid=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX" to "instanceUuid=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
       
       d.Edit "/etc/vmware-vpx/firstboot/vpxd-service-spec.prop", and update the cmreg.serviceid value from uppercase letters to lowercase letters.

        # vi /etc/vmware-vpx/firstboot/vpxd-service-spec.prop
        Change line "cmreg.serviceid=XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX" to "cmreg.serviceid=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
         
  2. Update the vpxd service registration with "lsdoctor" to reflect the changes made.

      a.Download lsdoctor tool from KB320837 and upload it into vCenter
      b.Run "python lsdoctor.py -r"

        Select option 3 when prompted (3. Replace individual service)
        Select the endpoint named "vcenterserver" - 
       
    **Please note that when selecting Option 3, Most templates are provided in the templates directory, and you would only need to select a template if it doesn’t exist.  If the template for your build of vCenter does not exist, you will be prompted to select one that most closely matches your system.

  3. Restart the vCenter services using the command below, and reregister the VASA provider
    service-control --stop --all && service-control --start --all

    Once all the services have started, check that required services are running
    service-control --status --all

  4. Fix the license tied to the vCenter

    a.Record the current vCenter license key.

    b.Obtain the current vCenter LDU ID with the command:
    # /usr/lib/vmware-vmafd/bin/vmafd-cli get-ldu --server-name localhost
    Output:   yyyyyyyy-yyyy-yyyy-yyyy-yyyyyyyyyyyy

    c.Remove the license associated with the current LDU ID by command:
    # /opt/likewise/bin/ldapdelete -r "cn=AssetEntity_<XXXXX>-<yyyyyyyy>,cn=LicenseService,cn=Services,dc=vsphere,dc=local" -D "cn=Administrator,cn=Users,dc=vsphere,dc=local" -W

Notes:

  • Adjust dc=vsphere,dc=local to reflect the environment's SSO domain
  • Replace XXXXXXXX with the vCenter service ID from step 1 in capital letters
  • Replace yyyyyyyy with the LDU ID from Step 4.c

        d. If the above step (c) fails to remove the license with an error similar to:

            ldap_delete: No such object (32) additional info: (9703)((MDB_NOTFOUND: No matching key/data pair    found

            Remove the vCenter license using KB - vCenter 6.x/7.x/8.x incorrectly displays the amount of licenses in use by ESXi hosts

        e. Restart the vCenter services and re-assign the license.

Additional Information

If SRM is used in the environment, please export the configuration data. When the vCenter GUID changes, SRM will break so we will need to unregister both of them then re-register and import the xml.

After implementing KB, if you receive the error: "Provider method implementation threw unexpected exception: vc server [VC UUID] is not available on the server." when trying to make changes to the content library, please open a ticket with support.


Powered by Wolken Software