Re-pointing vCenter to an existing domain fails during "Registering Infra services"
search cancel

Re-pointing vCenter to an existing domain fails during "Registering Infra services"

book

Article ID: 394409

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

The vCenter repoint operation to an existing domain fails with the below entries,

All Repoint configuration settings are correct; proceed? [Y|y|N|n]: Y

Starting License export                                ....Done
Starting Authz Data export                             ....Done
Starting Tagging Data export                           ....Done
Export Service Data                                    ....Done
Uninstalling Platform Controller Services              ....Done
Stopping all services                                  ....Done
Updating registry settings                             ....Done
Re-installing Platform Controller Services             ....Done
Registering Infra services                             ....Failed
Repoint failed. Restore from backup



In vCenter var/log/vmware/cloudvm/domain_consolidator.log you see entries similar to:

[YYYY-MM-DDTHH:MM:SS] INFO domain_consolidator Registering endpoint with id #######-#####-####-####-###2323590
[YYYY-MM-DDTHH:MM:SS] INFO domain_consolidator Changing domain names in spec file: /storage/domain-data/service-phase-data/specs/#######-#####-####-####-###2323590.spec
[YYYY-MM-DDTHH:MM:SS] INFO domain_consolidator Running command ['/usr/java/jre-vmware/bin/java', '-Djava.security.properties=/etc/vmware/java/vmware-override-java.security', '-cp', '/usr/lib/vmware/common-jars/commons-logging-1.1.3.jar:/usr/lib/vmware/common-jars/slf4j-api-1.7.30.jar:/usr/lib/vmware/common-jars/httpclient-4.5.3.jar:/usr/lib/vmware/common-jars/httpcore-4.4.6.jar:/usr/lib/vmware-sca/lib/lookup-client.jar:/usr/lib/vmware-sca/lib/*:/usr/lib/vmware-sca/lib', '-Dlog4j.configuration=tool-log4j.properties', 'com.vmware.vim.lookup.client.tool.LsTool', 'register', '--url', 'https://vCenter_FQDN:443/lookupservice/sdk', '--user', 'Administrator@<your_domain>.local', '--id', '#######-#####-####-####-###2323590', '--spec', '/storage/domain-data/service-phase-data/specs/#######-#####-####-####-###2323590.spec', '--password', '*****CENSORED*****']
[YYYY-MM-DDTHH:MM:SS] INFO domain_consolidator lstool register services failed: 1
[YYYY-MM-DDTHH:MM:SS] INFO domain_consolidator Failed to register services during repointing
[YYYY-MM-DDTHH:MM:SS] INFO domain_consolidator ESC[91m FailedESC[0m
[YYYY-MM-DDTHH:MM:SS] ERROR domain_consolidator Failed to Re-install PSC services
[YYYY-MM-DDTHH:MM:SS] INFO domain_consolidator Embedded Domain Repoint Service Command Phase Failed. Please check logs
[YYYY-MM-DDTHH:MM:SS] INFO domain_consolidator Failed executing <cis.service_data.DcServicesCommand object at #########>
[YYYY-MM-DDTHH:MM:SS] ERROR domain_consolidator Re-pointing operation has failed during execution mode.
[YYYY-MM-DDTHH:MM:SS] INFO domain_consolidator ESC[91mRepoint failed. Restore from backupESC[0m

 

 

In vCenter var/log/vmware/lookupsvc/lookupserver-default.log you see entries similar to :

[YYYY-MM-DDTHH:MM:SS] pool-2-thread-5  ERROR com.vmware.vim.lookup.util.ValidateUtil] Invalid certificate
[YYYY-MM-DDTHH:MM:SS] pool-2-thread-5  ERROR com.vmware.vim.lookup.vlsi.util.VmodlEnhancer] Invalid certificate
java.lang.IllegalArgumentException: Invalid certificate
        at com.vmware.vim.lookup.util.ValidateUtil.logAndThrow(ValidateUtil.java:311) ~[lookupservice-lib-0.0.1-SNAPSHOT.jar:?]
        at com.vmware.vim.lookup.util.ValidateUtil.validateCertificate(ValidateUtil.java:281) ~[lookupservice-lib-0.0.1-SNAPSHOT.jar:?]
        at com.vmware.vim.lookup.util.ValidateUtil.validateBase64EncodedCertificate(ValidateUtil.java:296) ~[lookupservice-lib-0.0.1-SNAPSHOT.jar:?]
        at com.vmware.vim.lookup.ServiceRegistrationTypes$Endpoint.validate(ServiceRegistrationTypes.java:936) ~[lookupservice-lib-0.0.1-SNAPSHOT.jar:?]
        at com.vmware.vim.lookup.ServiceRegistrationTypes$MutableServiceInfo$Builder.validate(ServiceRegistrationTypes.java:261) ~[lookupservice-lib-0.0.1-SNAPSHOT.jar:?]
        at com.vmware.vim.lookup.ServiceRegistrationTypes$CommonServiceInfo$Builder.validate(ServiceRegistrationTypes.java:462) ~[lookupservice-lib-0.0.1-SNAPSHOT.jar:?]
        at com.vmware.vim.lookup.ServiceRegistrationTypes$CreateSpec$Builder.create(ServiceRegistrationTypes.java:546) ~[lookupservice-lib-0.0.1-SNAPSHOT.jar:?]
        at com.vmware.vim.lookup.vlsi.util.VmodlUtil.fromVmodl(VmodlUtil.java:222) ~[lookupservice-impl-0.0.1-SNAPSHOT.jar:?]
        at com.vmware.vim.lookup.vlsi.ServiceRegistrationImpl$1.call(ServiceRegistrationImpl.java:93) ~[lookupservice-impl-0.0.1-SNAPSHOT.jar:?]
        at com.vmware.vim.lookup.vlsi.ServiceRegistrationImpl$1.call(ServiceRegistrationImpl.java:63) ~[lookupservice-impl-0.0.1-SNAPSHOT.jar:?]
        at com.vmware.vim.lookup.vlsi.util.VmodlEnhancer.invokeVmodlMethod(VmodlEnhancer.java:67) [lookupservice-impl-0.0.1-SNAPSHOT.jar:?]
        at com.vmware.vim.lookup.vlsi.ServiceRegistrationImpl.create(ServiceRegistrationImpl.java:63) [lookupservice-impl-0.0.1-SNAPSHOT.jar:?]
        at sun.reflect.GeneratedMethodAccessor284.invoke(Unknown Source) ~[?:?]
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_351]
        at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_351]
        at com.vmware.vim.vmomi.server.impl.InvocationTask.run(InvocationTask.java:99) [vlsi-server-7.0.0-SNAPSHOT.jar:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [?:1.8.0_351]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [?:1.8.0_351]
        at java.lang.Thread.run(Thread.java:750) [?:1.8.0_351]
Caused by: java.security.cert.CertificateExpiredException: certificate expired on 20#########48GMT+00:00
        at org.bouncycastle.jcajce.provider.X509CertificateObject.checkValidity(Unknown Source) ~[bc-fips-1.0.2.1.jar:1.0.2.1]

Cause

This issue occurs when the the lookup service registrations have an invalid certificate that doesn’t match the MACHINE_SSL_CERT on port 443 of the node. This mismatch is typically the result of an incomplete or failed certificate replacement process.

Resolution

  1. Download the lsdoctor tool from Broadcom’s knowledge base:
    Using the lsdoctor tool

    NOTE: Before using lsdoctor to make any changes, ensure you have taken proper snapshots of your SSO domain. This means that you must shut down all VCs or PSCs that are in the SSO domain at the same time, then snapshot them, and power them on again.  If you need to revert to one of these snapshots, shut all the nodes down, and revert all nodes to the snapshot. Failure to perform these steps will lead to replication problems across the PSC databases.

  2. Copy and extract the lsdoctor script onto the vCenter’s filesystem. (Not on the Replication partner)

    • Run the command:

      python lsdoctor.py -t

    • Enter the SSO administrator password when prompted.

  3. Post-execution actions

    • Restart all vCenter services on every node within the SSO site.

    • Re-register any external VMware solutions (e.g., SRM, vSphere Replication, NSX-V) that were previously associated with the affected node(s).

  4. Repoint the domain by following Repointing a Single vCenter Server

Additional Information



Powered by Wolken Software