vCenter linking in VCF Operations fails with error "There are no valid vcenter instances available to proceed with the linking process."

Products

• VMware vCenter Server
• VCF Operations

Issue/Introduction

• While attempting to create vCenter linking in VCF Operations, the following error is encountered “There are no valid vCenter instances available to proceed with the linking process.”

• Logs on VCF Operations located at /storage/vcops/log/vrops-vcenter-linking.log shows entries similar to:
  
  YYYYY-MM-DDThh:mm:ss INFO  [AsyncExecutor-2] com.vmware.vrops.vcenterlinking.manager.StateManager - Enabling linking for vCenters [10fd5b48-xxxx-xxxx-xxxx-f2806e9b0a8a, dffd3ae2-xxxx-xxxx-xxxx-295af405b603]
YYYYY-MM-DDThh:mm:ss INFO  [AsyncExecutor-2] com.vmware.vrops.vcenterlinking.manager.StateManager - Starting TRUST_ESTABLISHMENT state processing for vCenter 10fd5b48-xxxx-xxxx-xxxx-f2806e9b0a8a
YYYYY-MM-DDThh:mm:ss INFO  [AsyncExecutor-2] com.vmware.vrops.vcenterlinking.manager.StateManager - Initiating FetchIdentityParam task for vCenter 10fd5b48-xxxx-xxxx-xxxx-f2806e9b0a8a
YYYYY-MM-DDThh:mm:ss INFO  [AsyncExecutor-2] com.vmware.vrops.vcenterlinking.manager.StateManager - Updating vCenter 10fd5b48-xxxx-xxxx-xxxx-f2806e9b0a8a with Domain Id 9ca1a6b1-xxxx-xxxx-xxxx-79989c717750
YYYYY-MM-DDThh:mm:ss INFO  [AsyncExecutor-2] com.vmware.vrops.vcenterlinking.manager.StateManager - Initiating TrustEstablishmentParam task for vCenter dffd3ae2-xxxx-xxxx-xxxx-295af405b603
YYYYY-MM-DDThh:mm:ss ERROR [AsyncExecutor-2] com.vmware.vrops.vcenterlinking.manager.StateManager - TRUST_ESTABLISHMENT failed for the Vcenter Id: 10fd5b48-xxxx-xxxx-xxxx-f2806e9b0a8a with reason: ERROR_UNKNOWN
YYYYY-MM-DDThh:mm:ss INFO  [AsyncExecutor-2] com.vmware.vrops.vcenterlinking.manager.StateManager - Starting TRUST_ESTABLISHMENT state processing for vCenter dffd3ae2-xxxx-xxxx-xxxx-295af405b603
YYYYY-MM-DDThh:mm:ss INFO  [AsyncExecutor-2] com.vmware.vrops.vcenterlinking.manager.StateManager - Initiating FetchIdentityParam task for vCenter dffd3ae2-xxxx-xxxx-xxxx-295af405b603
YYYYY-MM-DDThh:mm:ss INFO  [AsyncExecutor-2] com.vmware.vrops.vcenterlinking.manager.StateManager - Updating vCenter dffd3ae2-xxxx-xxxx-xxxx-295af405b603 with Domain Id 9ca1a6b1-xxxx-xxxx-xxxx-79989c717750
YYYYY-MM-DDThh:mm:ss ERROR [AsyncExecutor-2] com.vmware.vrops.vcenterlinking.manager.StateManager - TRUST_ESTABLISHMENT failed for the Vcenter Id: dffd3ae2-xxxx-xxxx-xxxx-295af405b603 with reason: NO_AVAILABLE_PEERS
YYYYY-MM-DDThh:mm:ss ERROR [ServerConnection on port 10000 Thread 16] com.vmware.vrops.vcenterlinking.server.VcLinkingServer - Failed to update vCenter linking assignments
com.vmware.vrops.vcenterlinking.client.exception.LinkingVcenterLimitException: Number of vCenter resources not in limit.
  
  
• Logs on VCF Operations located at /storage/vcops/log/adapters/ManagementAdapter/ManagementAdapter_xx.log shows entries similar to:

YYYY-MM-DDThh:mm:ss ERROR ManagementAdapter 6346 [ops@4413 threadId="352312" threadName="TasksManager-TaskHandler-6627" instanceId="58"] [(58) com.vmware.adapter.management.components.vclinking.task.C.N] -  Exception while establishTrust
com.vmware.vapi.std.errors.AlreadyExists: AlreadyExists (com.vmware.vapi.std.errors.already_exists) => {
    messages = [LocalizableMessage (com.vmware.vapi.std.localizable_message) => {
    id = com.vmware.vcenter.trustmanagement.alreadyexists,
    defaultMessage = Trust object already exists for given domain ID (9ca1a6b1-xxxx-xxxx-xxxx-79989c717750),
    args = [9ca1a6b1-xxxx-xxxx-xxxx-79989c717750],
    params = <null>,
    localized = <null>
}],
    data = <null>,
    errorType = ALREADY_EXISTS
}

• You may observe the same DomainGUID value on all vCenter instances. This can be validated by running the following command on each vCenter:
  

  
  /opt/likewise/bin/lwregshell list_values '[HKEY_THIS_MACHINE\Services\vmafd\Parameters]' | grep -i DomainGUID

Environment

VCF 9.x
VCF Operations 9.x 

Cause

This issue occurs when the vCenter instances are still part of an Enhanced Linked Mode or were not properly deactivated/removed from ELM, resulting in residual configuration data that prevents successful linking operation.

Resolution

1. Follow the steps outlined in KB Deactivating Enhanced Linked Mode (ELM) in vCenter 9.0 (VVF) properly remove the vCenter instances from ELM. If this process fails, see: vCenter linking in VCF Operations fails with errors, "An error has occurred. Retry the linking process after sometime."
   
   
2. Confirm that each vCenter Server has a unique DomainGUID by executing the following command on all vCenter nodes:
   

   /opt/likewise/bin/lwregshell list_values '[HKEY_THIS_MACHINE\Services\vmafd\Parameters]' | grep -i DomainGUID

3. After completing the procedure, re-attempt the vCenter linking operation.

Additional Information

Group and Link Your vCenter Servers to Simplify Workload Management

Linking vCenter instances in VCF Operations

Splitting Enhanced Linked Mode (ELM)