Deployment requests are stuck in Pending approval state in XL profile configuration

Products

• VMware Aria Suite

Issue/Introduction

Symptoms:

• Deployment requests are stuck in Pending approval state, if approval happens after 2 or more days.
• The problem occurs on vRA environments deployed with XL profile.

Validate the deployments are stuck because of this specific problem

• Search for the following log patterns:

approval-service-app logs search for

22097-No event trace entry

ebs-app logs search for

Event trace entry [guid] for org [Optional[guid]] not found

Note: guid will be actual ids in GUID format.

Environment

VMware vRealize Automation 8.6.x
VMware vRealize Automation 8.8.x
VMware vRealize Automation 8.7.x

Cause

For storage optimization purposes in a XL profile environment, by default, EBS entities linked to long running extensibility tasks (approvals included) are cleaned up aggressively after 2 days. For some approvals, this time period is not enough and the deployment eventually becomes stuck.

Resolution

This issue is resolved in vRealize Automation 8.8.2.

Workaround:

Prerequisites

• Please take simultaneous non-memory snapshots of each virtual appliance(s) in the cluster.

Note: This can be performed from the vRSLCM instance that manages this environment.

• You have access to root user and password.
• You have SSH or console access to each virtual appliance.

Procedure

1. SSH / PuTTy into one vRA virtual appliance in the cluster
2. Run the following command

   vracli cluster exec -- sh -c "base64 -d <<< '/Td6WFoAAATm1rRGAgAhARYAAAB0L+Wj4AMmAQ5dABGICYgKg0ZqueSJRBesI0wUCxSfsgc6v9/yWcQHf88y0WNdrjOeUAb+UdcoZZ4qDujORMcpnTHL7xmIvYBNyw5g7Lkv5Yk5VZB929vXRbMIhpOqXl9DT1rqtSWx3Ai4kFoUpPJ5xkoOoaLfZelhCpv+5NXOigBPFyjWP0VK6CYkiGQNp22tFZ1Est7cGqJzSjEk8QWeAsVZX8ToW854BmCQVZF5U8rvIm/jjY8SNC7PajkPnns42ul2HGg0xJAWEQEad4suicPpppt3ngCxAtDY98YUYSkjJ+JoNSTOnlU1BePUb3Cob7f9CVi9YBsKqfrKhUr2y3/yre4dbdxZ2mBwhKpl7aYFQWo4pBq/AAAAAPSwD2EsdmTbAAGqAqcGAAA76CPVscRn+wIAAAAABFla' | xz -d | sh -"

3. Redeploy vRA services by running

   /opt/scripts/deploy.sh

This will increase the EBS retention time to 8 days.

Note: This may have storage implications since the ebs-db will grow.