Provisioning VM from vRA portal hangs at 96% with an active cross cluster vmotion in progress

Products

• VMware Aria Suite
• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• Provisioning VM from vRA will hang at 96% and will not progress until any cross cluster vmotions occurring in the environment completes
• You see this issue when template from where the vRA clones the VM is present in same datastore where the Storage vMotion is initiated (Source Datastore).
• Once the storage vMotion jobs is complete, all the deployments completes immediately.

Environment

VMware vRealize Automation 7.2.x
VMware vCenter Server 6.x
VMware vSphere ESXi 6.0

Cause

A cross-Cluster vMotion/Storage vMotion that is happening take's a statelock against both source and destination compute resources, in this case clusters.
This then blocks the Reconfigure from the vRA provisioning side, which needs an EXCLUSIVE statelock against the Cluster to complete its operation hence the provisioning operation will wait till storage vmotion complete. vRA will then mark the provisioning operation as finished.

Resolution

This issue is resolved in vCenter Server 6.7 Update 1.

Workaround:
This was a fundamental locking mechanism provided by vCenter in earlier versions. As a workaround wait until any cross-Cluster vMotion/Storage vMotions are completed and then provisioning operations will complete.