Since upgrading to 8.18.x we see the following when trying to run actions or workflows in Aria Orchestrator
We would see similar in the vco-server-app.log
2024-07-22T04:03:42.697Z WARN vco [host='vco-app-7bfd6d5667-5b54t' thread='licenseComplianceScheduler-1' user='' org='' trace=''] {} com.vmware.o11n.service.license.retriever.LicenseComplianceRetriever - Error occurred during license compliance retrieval. com.vmware.o11n.service.license.RetrieveLicenseException: Request to automation license endpoint failed.
2024-07-22T04:03:42.701Z WARN vco [host='vco-app-7bfd6d5667-5b54t' thread='licenseComplianceScheduler-1' user='' org='' trace=''] {} com.vmware.o11n.service.license.task.LicenseComplianceTask - License compliance state is restricted.
2024-07-22T04:08:53.369Z ERROR vco [host='vco-app-7bfd6d5667-5b54t' thread='http-nio-8280-exec-118' user='' org='' trace='-'] {} com.vmware.o11n.web.common.BaseController - Error invoking REST [unknown] ch.dunes.util.DunesServerException: ch.dunes.util.OperationNotEntitledException: Requested operation is not permitted due to no valid license entitlement
The observed failures happen intermittently.
Some calls to the orchestrator are failing with errors after the license entitlement restriction checks is reported as restricted. The license check in turn has failed due to usage of an expired token used for authentication. After several minutes the expired token is refreshed and the license restriction is lifted resulting in continuing normal operation.
For 8.18.0 and 8.18.1 we have the following workaround: Run the following command to apply the license check system property:
vracli vro properties set --key com.vmware.o11n.license.check.automation-endpoint.enabled --value false
Intermittent failures of running workflows and actions with an exception - "ClientResponse has erroneous status code: 403 Forbidden"
Intermittent failures of running workflows and actions with an exception - "ClientResponse has erroneous status code: 403 Forbidden" on Automation Orchestrator deployments authenticated with VMware Aria Automation.
Workaround: Run the following command to apply the license check system property:
vracli vro properties set --key com.vmware.o11n.license.check.automation-endpoint.enabled --value false
NOTE: This vracli command will cause a restart of the vco service