Diagnostics for VMware Cloud Foundation: Virtual Machine Operations

Products

• VCF Operations

Issue/Introduction

VMware Cloud Foundation Operations provides information on virtual machine operations that deploy, power on/off, reconfigure, and destroy virtual machines and vApps. This KB helps you troubleshoot problems with this feature itself and problems with virtual machine operations. 

The information on virtual machine operations is gathered from tasks executed by vCenter. Every 5 minutes, VCF Operations connects to all the integrated vCenters, and fetches the vCenter tasks and related events. It extracts relevant information for each virtual machine operation failure, and persists it. When the UI page is opened, the UI reports details about these failed operations and provides links to additional information on the failures. 

The feature monitors the following virtual machine operations. The terms used below are those shown in the vSphere Client UI. For further understanding of the reasons for a failure, every failure report includes a link to the vCenter API documentation for the API that was invoked. 

Create Operations Failures

• Apply DRS clone recommendation 
• Create New Virtual Machine
• Clone vApp
• Clone Virtual Machine or Template
• Deploy OVF 
• Mark a template as Virtual Machine
• Register Virtual Machine

Power Operation Failures

• Apply DRS power on recommendation 
• Power on, off, and and suspend virtual machine 
•  Initiate guest OS shutdown
•  Start, stop, and suspend vApp
•  Initiate guest OS standby
•  Hard stop virtual machine
•  Reset Virtual Machine
•  Reboot Guest

Delete Operation Failures

• Delete Virtual Machine
• Unregister and Delete Virtual Machine
• Delete and unregister vApp 

Reconfigure Operation Failures

• Reconfigure Virtual Machine
• Attach or detach a virtual disk
• Update VM Hardware version 
• Guest OS operations (VMware Tools, Guest os Customizations etc) 

Environment

Operations for VMware Cloud Foundation 9.0

Resolution

Problems with the Virtual Machine Operations Dashboard

Problem: the Virtual Machine Operations page is not reporting any virtual machine operations or is missing operations from certain vCenters. 

Resolution: VCF operations collects the data for this feature from each vCenter instance. This data collection is implemented by vCenter adapters. Navigate to Administration->Integrations in the VCF Operations UI and verify there is an active integration for each vCenter. If any VC adapters are in the stopped state, start collection. If any adapters are reporting a collection warning or an error, address the problem.

Problem: the information reported in the VCF Operations UI about a failure is not sufficient to understand why the failure occurred. 

Resolution: for some failures, vCenter reports additional information. In the vSphere Client, navigate to the Task Console, expand the row for the task you wish to investigate, and review the Error Stack. If this information is insufficient, use the Infrastructure Operations > Analyze > Logs page to search for the log statements issued by the operation. Follow instructions in KB https://knowledge.broadcom.com/external/article/376010.

 

For troubleshooting and known issues related to virtual machine provisioning please refer to the following KBs.  

• Cannot deploy a virtual machine from a template because the template VMDK file is locked
• Virtual machine deployments fail with error "Provisioning operation failed. Cannot deploy library item..."
• Virtual machine disk provisioning type changes unexpectedly

 

For troubleshooting general virtual machine operation issues caused by file locks please refer to the following KBs.  

• Investigating virtual machine file locks on ESXi hosts
• Investigating virtual disk file locks on vSAN

Additional Information

Virtual machine and vApp failures can be caused by other common causes in addition to file lock issues. Use the filters provided in the virtual machine operations page to identify topology-related common causes.