Orchestrator workflow to create Cloud Zone fails when called via Service broker

Products

• VMware Aria Suite

Issue/Introduction

Symptoms:

• An Orchestrator workflow to create Cloud Zones via the Iaas api fails when called via Service broker on versions prior to Aria Automation 8.12
• Checking the vCO logfiles under /var/log/services-logs/prelude/vco-app/file-logs/vco-server-app.log you can see the call is made with the vRO gateway user and an orgless client token:

2023-10-10T10:36:26.635Z INFO vco [host='vco-app-5777fffd78-j2h5j' thread='WorkflowExecutorPool-Thread-1' user='vro-gateway-VlYTT1afXsuLvrO8' org='-' trace='bcd5da30953e4e7b9f378b4d6674078d'] {|__SYSTEM|vro-gateway-VlYTT1afXsuLvrO8:Create Cloud Zone:28046eb4-4e5c-4aa5-8293-74793f1a49de:token=9815eb4c-25b5-49c8-9d4c-bc91dee8d035:context=bcd5da30953e4e7b9f378b4d6674078d} com.vmware.o11n.plugin.vra.svc.VraClientFactory - Invoking POST request https://<vRAFQDN>/iaas/api/zones?apiVersion=2021-07-15 

• The tango-vro-gateway logs located under /var/log/services-logs/prelude/tango-vro-gateway-app/file-logs/tango-vro-gateway-app.log contain a ResourceNotFoundException similar to:

2023-09-28T07:40:29.429Z ERROR tango-vro-gateway [host='tango-vro-gateway-app-5f49f8dd6b-gs6v8' thread='reactor-http-epoll-3' user='' org='' trace='<traceID>' parent='' span='<spanID>'] reactor.core.publisher.Operators.error:315 - Operator called default onErrorDropped
    reactor.core.Exceptions$ErrorCallbackNotImplemented: com.vmware.automation.vro.gateway.common.ResourceNotFoundException: Cannot find a custom resource definition for orgId: <OrgID>, projectId: <ProjectID>, externalType: VRA:Zone
    Caused by: com.vmware.automation.vro.gateway.common.ResourceNotFoundException: Cannot find a custom resource definition for orgId: <OrgID>, projectId: <ProjectID>, externalType: VRA:Zone

• The same workflow executes successfully when called directly in Orchestrator.

Environment

VMware vRealize Orchestrator 8.x

Cause

The error is in the token with which vro calls the IaaS API. In vRA version 8.10 the code is calling with orgless client token. While in 8.12 and later it is calling with the user token.

Therefore in 8.10 the cloud zone is created without the org tenant link. When the UI queries the backend with a user token, the zone is filtered out since it doesn't match the org of the user.

Resolution

To resolve the issue upgrade to Aria Automation 8.12 which enables the user token delegation setting