Deleting additional disks fails with timeout 'Request timed out after 120 minutes' in Aria Automation 8.11.2 & higher for deployments onboarded before version 8.6.1

Products

• VMware Aria Suite

Issue/Introduction

Symptoms:

• Deleting additional disks from deployments onboarded before version 8.6.1 fails with timeout 'Request timed out after 120 minutes' in Aria Automation 8.11.2 & higher
• The disk is removed from the virtual machine in vCenter but is not deleted from Onboarded deployment.
• The provisioning service logs have a warning similar to:

WARN provisioning [host='provisioning-service-app-<id>' thread='xn-index-queries-14' user='' org='' trace='<trace_id>' parent='' span='<span_id>'] c.v.a.r.c.ComputeOperationTaskService.log:454 - [8282/provisioning/requests/compute-resource-operations/<resource_id>] Request timed out after 120 minutes. Please configure project request timeout parameter for long running resource requests. Error: com.vmware.xenon.common.LocalizableValidationException: Request timed out after 120 minutes. Please configure project request timeout parameter for long running resource requests.
at com.vmware.admiral.service.common.AbstractTaskStatefulService.handleFailOnTimeoutAction(AbstractTaskStatefulService.java:2654)
at com.vmware.admiral.service.common.AbstractTaskStatefulService.handleSpecialAction(AbstractTaskStatefulService.java:2524)

 

Environment

VMware vRealize Automation 8.x

Cause

The disc_desc_links field in the ComputeDescription is not populated on onboarded machines prior to vRealize Automation version 8.6.1. Deleting such disks occurs without issue until the 8.11.2 release. A clean up script was introduced in 8.11.2 to remove the disk description link of the disk to be removed from compute description which results in the timeout error described.

Resolution

The issue is scheduled to be resolved in the Aria Automation 8.16.1 release

Workaround:

To workaround the issue perform the following steps:
 

Note: Before proceeding with either of the options below take snapshots of the Aria Automation appliances.
 

For Deployment without a failed Remove Disk operation performed:

1. In the Aria Automation UI under Cloud Assembly > Resources > Deployments select the affected onboarded machine associated with the disk in the topology diagram and expand the custom properties field in the lower right hand panel. Document the value of the resourceDescLink property for use in the next steps

2. Login to the Aria Automation appliance via SSH with the root user.
3. Connect to provisioning service database:

vracli dev psql provisioning-db

4. Query the compute_description table using the resourceDescLink property value documented step 1.

select * from compute_description where document_self_link = '<resourceDescLink property value>';

For example:

select * from compute_description where document_self_link = '/resources/compute-descriptions/29c73b3d-9000-497e-8ee3-eda191d50290';

The output for disk_desc_links in the query result is Null. For example ( disk_desc_links | )

5. Update the disk_desc_links with an Empty array using the below query:

update compute_description set disk_desc_links = '[]' where document_self_link = '<resourceDescLink property value>';

The day 2 remove disk action should now succeed.
 

For Deployment with a failed Remove Disk operation already performed:

1. Generate a bearer token for authentication with the IaaS api
2. Attach the disk which was tried earlier to be removed from VM using the IaaS api

POST https://<AriaAutomation-FQDN>/iaas/api/machines/{machineid}/disk

The add a disk api is documented in the swagger api documentation .

3. Perform the steps 1-5 above to manually update the database for disk_desc_links to have an Empty array instead of a null value.

The day 2 remove disk action should now succeed.