Intermittent failures of running workflows and actions with an exception - "ClientResponse has erroneous status code: 403 Forbidden".

Products

• VCF Operations/Automation (formerly VMware Aria Suite)

Issue/Introduction

• Workflows and tasks normal flow could be disrupted and would need to be retriggered.
• You may see this intermittent error message during VM creation in the UI:

  Failed to process ebs event: SubscriberID: vro-gateway-<ID>, RunnableID: <ID> and SubscriptionID: <ID> failed with the following error: Workflow run [<ID>] completed with error [Run tracking failed with an exception: ClientResponse has erroneous status code: 403 Forbidden]

Environment

VMware Aria Automation 8.x

VMware Aria Orchestrator 8.x

Cause

Operations failures are due to intermittent license check failures.

Resolution

Upgrade to the latest version of Automation or Orchestrator.

For Orchestrator 8.18.0 with Aria Automation authentication, you can use the following workaround instead:

Apply a system property on the Orchestrator service:

1. Snapshot or Backup your environment in case you need to revert the changes.
2. Establish an ssh session to the Orchestrator appliance.
3. Apply the following system property using the vracli command:

vracli vro properties set --key com.vmware.o11n.license.check.automation-endpoint.enabled --value false 

The configuration of the above system property will cause an Orchestrator restart.

Note: On cluster environments the above command needs to be applied only on one of the nodes.

Note: For embedded Orchestrator, run this command on the Aria Automation node.