Service Mesh creation is failing with error "VSM response error (500): vCenter Connection is not available"

Products

• VMware HCX

Issue/Introduction

• HCX Service Mesh deployment will fail with below error :
  

  Service Mesh modification failed. Interconnect Service Workflow PrepareFleetPools failed. Error: Interconnect Service Workflow PrepareFleetNetwork failed. Error: VSM response error (500): vCenter Connection is not available.

• HCX UI shows the below error :
  
• NSX-v is configured at HCX Source side.
• HCX Admin 9443 UI shows vCenter/SSO/NSX as healthy.

• HCX logs will show below logging : /common/logs/admin/app.log


  <timestamps> UTC [InterconnectService_SvcThread-297, SM:servicemesh-####-####, IX:####-####, J:####, , TxId: ####-####-####] ERROR c.v.v.h.a.n.e.NetworkSecurityErrorHandler- Response error xml : <?xml version="1.0" encoding="UTF-8"?>
  <error><errorCode>500</errorCode><details>vCenter Connection is not available.</details><moduleName>core-services</moduleName></error>
  <timestamps> UTC [InterconnectService_SvcThread-297, SM:servicemesh-####-####, IX:####-####, J:####, , TxId: ####-####-####] ERROR c.v.v.h.s.i.PrepareFleetNetwork- PrepareFleetNetwork failed in state CREATE_FLEET_NETWORK. Error: VSM response error (500): vCenter Connection is not available..
  com.vmware.vchs.hybridity.adapters.nsx.error.VsmException: VSM response error (500): vCenter Connection is not available.
  <timestamps> UTC [InterconnectService_SvcThread-288, SM:servicemesh-####-####, IX:####-####, J:####, , TxId: ####-####-####] ERROR c.v.v.h.s.i.PrepareFleetPools- PrepareFleetNetwork failed, errorCode:null. stacktrace:null, errorMessage:VSM response error (500): vCenter Connection is not available.

   

• NSX-v logs will show below logging : /logs/management_service/vsm.log


  <timestamps>  INFO DefaultVcConnectionKeepaliveThread DefaultVcConnectionKeepaliveThread:151 - - [nsxv@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Could not get VC Connection:VcConnectionNotAvailableException: core-services:500:vCenter Connection is not available.:com.vmware.vim.vmomi.core.exception.CertificateValidationException: Server certificate chain is not trusted and thumbprint verification is not configured
  <timestamps> ERROR pool-54-thread-1 SoapBindingImpl:258 - The SSL certificate of STS service cannot be verified
  com.vmware.vim.sso.client.impl.ssl.UntrustedSslCertificateException: The SSL certificate of STS service cannot be verified

 

Environment

VMware HCX
NSX-v at Source HCX

Cause

This issue was caused due to the vCenter registration on NSX-v being down or unavailable.

Resolution

Please ensure that the NSX Compute Manager's Connection Status is showing as Up.

If you believe you have encountered this issue and have NSX-v Extended Support, please open a NSX-v support case with Broadcom Support and refer to this KB article.
For more information, see Creating and managing Broadcom support cases

Workaround :

• Validate HCX Network Profiles do not use NSX port groups and only use Distributed Port Groups.
• Remove NSX-v from source HCX Admin 9443 UI.
  
  
  

Additional Information

Configuring the NSX SSO Lookup Service fails
NSX Requirements for HCX integration
End of Availability-NSX Data Center for vSphere (NSX-v)


Enforcing NSX configuration to the destination HCX Cloud Manager
With 4.11 release, site pairing with HCX Manager is not permitted without registering a valid NSX Manager at the target/destination side. Users will be notified to ensure NSX configuration is done for destination HCX cloud manager system under the HCX Appliance Management (:9443 UI) interface.