Collecting diagnostic information from VMware Aria Operations

Products

• VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

VMware Technical Support routinely requests diagnostic information from you when a support request is handled. This diagnostic information contains product specific logs and configuration files from the host on which the product is run. The information is gathered using a specific script or tool for each product.

Environment

Aria Operations 8.12x and later

Resolution

Quick links:

• Generate a support bundle from the User Interface (UI)
• Generate a support bundle through the console or command line
• vRealize Operations Manager 7.0 and later
• Generate a support bundle from a Cloud Proxy node
• Generate Debug Logs for Management Packs

Generate a support bundle from the User Interface (UI)

Use the steps below to generate a support bundle (log bundle) from the User Interface of VMware Aria Operations (on-prem).

1. Log in to the VMware Aria Operations UI.
2. Navigate to the Administration > Support Bundles page, Administration > Control Panel > Support Bundles page for 8.18.x, or Administration > Support > Support Bundles page for older versions.
3. Click Add and select the desired bundle type:

• Light support bundle - includes 24 hours of logs.
• Full support bundle - includes full log and charts.

5. Select the nodes to include in the support bundle, then click OK to generate the support bundle.

After the bundle has completed, you may download the file, then upload it to VMware Technical Support.  For more information, see Uploading files to cases on the Broadcom Support Portal.

Note: The slice information bundle download triggers a pop-up and some browsers may block this.  Ensure to disable your pop-up blocker for the VMware Aria Operations site to ensure that all information is downloaded appropriately.

Note: If the cluster is offline and the VMware Aria Operations product UI is inaccessible, a log bundle can still be generated from the Admin UI if it is still accessible.  Otherwise, proceed to the next section.
 

Generate a support bundle through the console or command line

If the user interface is inaccessible, you can generate the support bundle (log bundle) through the console or command line of VMware Aria Operations (on-prem).

Notes:

• Generating a support bundle through the console or command line is done on a per node basis.  Repeat this process on all nodes, or Cloud Proxies that you wish to collect a support bundle from.
• SSH is not enabled by default. For more information on enabling SSH, see Enabling SSH access in vRealize Operations.
• The generateSupportBundle.py script runs with these default options:

action: -a=create
filter: -f=1,2,3 (all logs,configs and support dumps; clusterInfo not included)
timestamp: -t=1394913286423 (current time in milliseconds)
Option: m=false so all log files will be downloaded, so support bundle may be large.

• By default, a full log bundle is generated.  To Generate a light log bundle, add the -m flag to the command.

Example: python generateSupportBundle.py -m

• All available options for running the script can be viewed by running this command:

                        python generateSupportBundle.py -h

vRealize Operations Manager 7.0 and later

Use the following steps for  vRealize Operations Manager 7.0 - 7.5, vRealize Operations 8.0 - 8.10, and VMware Aria Operations 8.12 and later.

1. Log into the primary node as root via SSH or Console.
2. Navigate to the tools directory by using this command:

cd /usr/lib/vmware-vcopssuite/utilities/bin/

3. Run the generateSupportBundle.py script using this command:

python generateSupportBundle.py

4. The resulting support bundle will be placed in the /usr/lib/vmware-vcops/support/timestamp directory.

Generate a support bundle from a Cloud Proxy node

vRealize Operations 8.5 and Later

Use the following steps to generate a support bundle from a VMware Aria Operations Cloud Proxy node, available in vRealize Operations 8.5 and later.

1. Log into the Cloud Proxy as root via SSH or Console.
2. Run the following command to generate a support bundle:

cprc-cli -sb IS_HEAVY

Note: Replace IS_HEAVY with true or false (case sensitive) depending if you want the journal.ctl logs included in the support bundle.  

Example: cprc-cli -sb true or cprc-cli -sb false

3. The resulting support bundle will be placed in the following location:

• • /storage/core/vmware-vrops-cprc/support
    
    

Aria Operations 8.18.x and Later

Use the following steps to generate a support bundle from a VMware Aria Operations Cloud Proxy node, available in Aria Operations 8.18.x and later.

1. Log into the Aria Operations Admin UI.
2. Click Support > Support Bundles (Cloud Proxy)
3. Click "Create Support Bundle" (the + icon)
4. Highlight and place a check mark next to the Cloud Proxy you would like to generate a bundle for, click OK.  Then click OK on the Info Message. 
5. Once the generation status is set to "Completed", select the bundle and click "Download Support Bundle" (The down arrow icon).
6. The resulting support bundle will be placed your downloads folder.

Generate Debug Logs for Management Packs

Applicable for: Management Pack for ServiceNow, Management Pack for Microsoft SQL Server, Management Pack for PostgreSQL, Management Pack for MySQL, Management Pack for Oracle Database, Management Pack for MongoDB

To generate debug logs for the applicable management packs, there are two options:

1. Download and run Pavo
   1. Download Pavo from Box: https://broadcom.box.com/v/pavo-debug-logging-tool
   2. Copy pavo.py to the VCF Operations node or Cloud Proxy where the relevant adapter instance is running
   3. SSH into the node or Cloud Proxy and run python pavo.py, selecting the options relevant to your scenario
2. Manually configure DEBUG logs and capture tail
   1. SSH into the VCF Operations node or Cloud Proxy where the relevant adapter instance is running
   2. In $VCOPS_BASE/user/conf/collector/log4j2.properties, take note of the current value of rootLogger.level so that it can be reset later.
   3. Set rootLogger.level to DEBUG and save log4j2.properties. It may take a minute or two for the change to take effect.
   4. Change to the relevant adapter directory:
      cd $VCOPS_BASE/user/log/adapters/[adapterkey]
   5. Tail the relevant adapter log for the requested number of collections:
      tail -F XXXXAdapter_YY.log | tee longrun-YYYYMMDDHHMMSS-XXXXAdapter_YY.log
   6. When the tail has captured the required collections, cancel it (ctrl-c).
   7. Reset rootLogger.level in $VCOPS_BASE/user/conf/collector/log4j2.properties to the original value

Additional Information

Uploading files to cases on the Broadcom Support Portal
Enabling SSH access in vRealize Operations