Could not connect to one or more vCenter Server systems:https://vcenter.fqdn:443/sdk
Article ID: 397912
Updated On:
Products
Issue/Introduction
- vCenter instances linked (ELM) are not visible to one another, resulting in their absence from the vCenter UI.
- Could not connect to one or more vCenter Server systems:
https://<vcenter_fqdn>:443/sdkis seen on vCenter UI - vmdird replication is healthy as follows
<user>@vcsa1 [ ~ ]# /usr/lib/vmware-vmdir//bin/vdcrepadmin -f showpartnerstatus -h localhost -u administratorpassword:Partner: <vcenter_fqdn> Host available: YesStatus available: YesMy last change number: #####Partner has seen my change number: #####Partner is 0 changes behind.
- There are no duplicate service registrations as described in this article: "Error: "Could not connect to one or more vCenter Server Systems" in the vSphere Client"
- /var/log/vmware/vsphere-ui/logs/vsphere_client_virgo.log shows the below error:
[YYYY-MM-DDTHH:MM:SS.000Z] [ERROR] wcp-plugins-notifier-1 c.v.v.plugin.notification.WcpServicePluginNotificationsListener Unable to subscribe to vc <vcenter_fqdn> (ff7#####-####-####-####-#########bcf), nodeId: 85######-####-####-####-##########62 java.util.concurrent.CompletionException: com.vmware.vcenter.apigw.security.AuthenticationException: Failed to log into [uri=http://localhost:1080/external-vecs/http2/<vcenter_fqdn>/443/apigw, sessionMgr=SessionManagerInfo [_sessionMgrSvcId=com.vmware.cis.session, _loginOpId=create, _logoutOpId=delete], ssoDomain=<domain.local>(f7e#####-####-####-####-##########416)]
at java.util.concurrent.CompletableFuture.encodeRelay(CompletableFuture.java:326) at java.util.concurrent.CompletableFuture.completeRelay(CompletableFuture.java:338) at java.util.concurrent.CompletableFuture.uniRelay(CompletableFuture.java:925) at java.util.concurrent.CompletableFuture$UniRelay.tryFire(CompletableFuture.java:913) at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:488) at java.util.concurrent.CompletableFuture.completeExceptionally(CompletableFuture.java:1990) at com.vmware.vcenter.apigw.session.backend.authenticator.impl.VapiServiceAuthenticator$LoginAsyncHandle.setError(Unknown Source) at com.vmware.vapi.internal.protocol.client.msg.json.JsonApiProvider$2.setError(JsonApiProvider.java:681) at com.vmware.vapi.internal.protocol.client.msg.json.JsonApiProvider$ResponseCallbackImpl.setError(JsonApiProvider.java:473) at com.vmware.vapi.internal.protocol.client.msg.json.JsonApiProvider$ResponseCallbackImpl.failed(JsonApiProvider.java:322) at com.vmware.vapi.internal.protocol.client.msg.json.JsonApiProvider$1.failed(JsonApiProvider.java:515) at com.vmware.vapi.internal.protocol.client.rpc.http.handle.NioSingleResponseConsumer.responseCompleted(NioSingleResponseConsumer.java:67) at com.vmware.vapi.internal.protocol.client.rpc.http.handle.NioDecoratorConsumer.responseCompleted(NioDecoratorConsumer.java:45) at org.apache.http.impl.nio.client.MainClientExec.responseCompleted(MainClientExec.java:383) at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:173) at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:448) at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:338) at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265) at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:87) at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:40) at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114) at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162) at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337) at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315) at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276) at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104) at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591) at java.lang.Thread.run(Thread.java:750)
Caused by: com.vmware.vcenter.apigw.security.AuthenticationException: Failed to log into [uri=http://localhost:1080/external-vecs/http2/<vcenter_fqdn>/443/apigw, sessionMgr=SessionManagerInfo [_sessionMgrSvcId=com.vmware.cis.session, _loginOpId=create, _logoutOpId=delete], ssoDomain=<domain.local>(f7######-####-####-####-########416)] ... 22 common frames omitted Caused by: com.vmware.vapi.client.exception.InvalidSslCertificateException: HTTP response with status code 526 (enable debug logging for details) at com.vmware.vapi.internal.protocol.client.rpc.http.ApacheHttpUtil.validateHttpResponse(ApacheHttpUtil.java:97) at com.vmware.vapi.internal.protocol.client.rpc.http.handle.NioSingleResponseConsumer.responseCompleted(NioSingleResponseConsumer.java:61) ... 16 common frames omitted
- Executing the following command lists expired certificate/s
for i in $(/usr/lib/vmware-vmafd/bin/vecs-cli store list); do echo STORE $i; /usr/lib/vmware-vmafd/bin/vecs-cli entry list --store $i --text | egrep "Alias|Not After"; doneSTORE MACHINE_SSL_CERTAlias : __MACHINE_CERTNot After : MMM DD HH:MM:SS YYYY GMT ----> Date in the pastSTORE TRUSTED_ROOTSAlias : ######aeaadd5c3e3####c7036f809c4cf######Not After : MMM DD HH:MM:SS YYYY GMT
Environment
- vCenter Server 7.X
- vCenter Server 8.X
Cause
Expired Machine SSL causing SSL Handshake error between one or more vCenters.
Resolution
- Replace the expired machine SSL using one of the below methos depends on VMCA or Custom Signed certificate. Reference below.
Feedback
