Alert: Adapter instance object has error status on VCF object(s)

Products

• VCF Operations/Automation (formerly VMware Aria Suite)
• VCF Operations

Issue/Introduction

• Receiving an alert 'Adapter instance object has error status' triggering on VCF object(s)
• Status of the adapter instance shows as OK (8.x) or Collecting (9.x)
• Attempting validate connection for the VCF adapter instance fails with error "Request is taking longer time than expected, please try again in sometime"
• Examining the VCF adapter logs in /storage/log/vcops/logs/adapters/VcfAdapter/VcfAdapter_##.log, the following error may appear:
  
  'Error occurred in InventoryProcessor. Reason: Request is taking longer time than expected, please try again in sometime com.vmware.vrops.adapter.vcf.exception.VcfException: Request is taking longer time than expected, please try again in sometime'
  
  Or
  
  WARN  VcfAdapter 2866681 [ops@4413 threadId="792" threadName="Collector worker thread 22" instanceId="76"] [(76) com.vmware.vrops.adapter.vcf.auth.AuthenticationManager.warn] - Exception occurred while connecting to url <https://SDDC_URL/v1/tokens>
com.vmware.vrops.adapter.vcf.exception.VcfException: Request is taking longer time than expected, please try again in sometime
        at com.vmware.vrops.adapter.vcf.util.VcfAdapterRestClient.executeWithRetry(VcfAdapterRestClient.java:167) ~[VcfAdapter.jar:?]
        at com.vmware.vrops.adapter.vcf.auth.AuthenticationManager.getNewAccessTokenDataWithRetry(AuthenticationManager.java:61) ~[VcfAdapter.jar:?]
        at com.vmware.vrops.adapter.vcf.auth.AuthenticationManager.getNewAccessTokenDataWithoutRetry(AuthenticationManager.java:48) ~[VcfAdapter.jar:?]
at com.vmware.vrops.adapter.vcf.helper.VcfAdapterHelper.extractAdapterSettings(VcfAdapterHelper.java:142) ~[VcfAdapter.jar:?]
        at com.vmware.vrops.adapter.vcf.helper.VcfAdapterHelper.initializeAdapterConfiguration(VcfAdapterHelper.java:89) ~[VcfAdapter.jar:?]
        at com.vmware.vrops.adapter.vcf.VcfAdapter.initAdapter(VcfAdapter.java:357) ~[VcfAdapter.jar:?]
        at com.vmware.vrops.adapter.vcf.VcfAdapter.onCollect(VcfAdapter.java:202) ~[VcfAdapter.jar:?]
        at com.integrien.alive.common.adapter3.AdapterBase.collectBase(AdapterBase.java:775) ~[vrops-adapters-sdk.jar:?]
at com.integrien.alive.common.adapter3.AdapterBase.collect(AdapterBase.java:547) ~[vrops-adapters-sdk.jar:?]
       
• Accessing VCF Operations >> Fleet Management >> Lifecycle >> VCF Instances >> Selecting VCF instance remains spinning as given below:
  
  
  
  
  

Environment

VMware Aria Operations 8.18.x
VCF Operations 9.x

Cause

Slow API queries from VCF adapter in VCF/Aria Operations

Resolution

Step 1:

If there is a temporary network slowness on these nodes, reboot the VCF Operations nodes or SDDC manager nodes. If the issue persists, proceed to step.2. 

Step 2:

Important: Take snapshots of each of the VCF/Aria Operations nodes before applying any of the steps below

How to take a Snapshot of VCF/Aria Operations

1. SSH to primary node as root
2. Stop collector service: systemctl stop collector
3. Create a copy of 'config.properties' file: 
   
   cp /usr/lib/vmware-vcops/user/plugins/inbound/VcfAdapter/conf/config.properties /usr/lib/vmware-vcops/user/plugins/inbound/VcfAdapter/conf/config.properties.bkp
   
   
4. Edit the file 'config.properties' file: vi /usr/lib/vmware-vcops/user/plugins/inbound/VcfAdapter/conf/config.properties 
   
   Note: The vi command, gets to the text editor. Use i from the keyboard to edit the file.
    
5. Modify the API connection timeout from current value to 120.
   
   For example:
   
   Before: "api.connection.timeout=30"
   After: "api.connection.timeout=120"
   
   Then input Esc from the keyboard along with wq! to save the change and exit the text editor.
   
   
6. Start the collector service: systemctl start collector
7. Repeat these steps on all collectors that manage the VCF adapters.

 

Step 3:

Important: Take snapshots of each of the VCF/Aria Operations nodes before applying any of the steps below

How to take a Snapshot of VCF/Aria Operations

1. Check if the Adapter Certificate is Valid. KB 315906 (How to renew an adapter certificate or clear the Expired Certificates banner in Aria Operations) 
2. If the Certificate is Valid, Remove and Re-add the Adapter Instance.