VMware Aria Operations collects data at irregular intervals on hosts where Kubernetes is deployed

Products

• VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

Symptoms : 

• There are intermittent metric collections, e.g., CPU Usage, for HostSystem objects collected from their Kubernetes vCenter,
• The issue does not occur on other non-Kubernetes vCenters. 

Environment

 

• VMware vRealize Operations : 8.6.4
• Kubernetes product version    : 1.21
• Kubernetes MP version           : 1.6.1

 

Cause

• vCenter collection exceeds designated collection interval due to known performance issue with queryNetworkHint calls to vCenter API taking over 5 minutes by itself.
• Starting from vROps 8.4 the API call will be skipped if the desired properties collection is disabled. 
• No workaround is available for up to vROps 8.3. 

NetworkSystem#queryNetworkHint : 

• This API is used to query the following TOR properties for hosts
  
  

Key	Localization
net|discoveryProtocol|cdp|managementIpAddress	Network|Discovery Protocol|Cisco Discovery Protocol|Management IP Address
net|discoveryProtocol|cdp|systemName	Network|Discovery Protocol|Cisco Discovery Protocol|System Name
net|discoveryProtocol|cdp|portName	Network|Discovery Protocol|Cisco Discovery Protocol|Port Name
net|discoveryProtocol|cdp|vlan	Network|Discovery Protocol|Cisco Discovery Protocol|VLAN
net|discoveryProtocol|cdp|mtu	Network|Discovery Protocol|Cisco Discovery Protocol|MTU
net|discoveryProtocol|cdp|hardwarePlatform	Network|Discovery Protocol|Cisco Discovery Protocol|Hardware Platform
net|discoveryProtocol|cdp|softwareVersion	Network|Discovery Protocol|Cisco Discovery Protocol|Software Version
net|discoveryProtocol|cdp|timeToLive	Network|Discovery Protocol|Cisco Discovery Protocol|Time To Live
net|discoveryProtocol|lldp|managementIpAddress	Network|Link Layer Discovery Protocol|Management IP Address
net|discoveryProtocol|lldp|systemName	Network|Link Layer Discovery Protocol|System Name
net|discoveryProtocol|lldp|portName	Network|Link Layer Discovery Protocol|Port Name
net|discoveryProtocol|lldp|vlan	Network|Link Layer Discovery Protocol|VLAN
net|discoveryProtocol|lldp|timeToLive	Network|Link Layer Discovery Protocol|Time To Live

Resolution

• A fix was implemented, which involved decoupling the NetworkSystem#queryNetworkHint API from the vCenter Adapter's main data collection.
• This has been fixed under vRealize Operations version 8.16 

Additional Information

Workaround :

• Disabling collection of the properties listed above in the active policy for the problematic host will resolve the issue.

Procedure to disable metrics/properties :

1.) Login to vRops product UI and click on "Configure" & choose "Policies"
2.) Go to effective policy of the resource.
3.) In the policy editor, click on "Metrics and Properties".
4.) Use the "Select Object Type" field to choose the object type you want to modify metrics for (e.g. Virtual Machine, Host System, etc.).
5.) A list of metrics/properties for that object type will show up.
6.) Find the metric/property in the list (List mentioned in cause section above)
7.) Click the dropdown in the "State" column
8.) Select "Disable" and click "Save"