Org Group creation fails in Tanzu Hub

Products

VMware Tanzu Platform VMware Tanzu Platform - Hub VMware Tanzu Platform - Cloud Foundry VMware Tanzu Platform Core

Issue/Introduction

When attempting to create an Organization (Org) Group in Tanzu Hub, the process fails and displays the following error:

2026-06-01T05:06:49.521Z[INFO]Assigning role 'organization_group_manager' to 'admin@test.org' across 1 foundations
2026-06-01T05:06:51.061Z[ERROR]Role 'organization_group_manager' assignment to 'admin@test.org' failed in 1 of 1 foundations
2026-06-01T05:06:51.595Z[ERROR]Foundation entity vrn/provider:TAS/instance:p-bosh-################/Organization:########-####-####-####-###########: status=MISSING, error=Unexpected error: Failed to parse GraphQL errors. Please check and ensure Platform Services tile is at latest version. The minimum version is 10.4
2026-06-01T05:06:52.127Z[ERROR]Foundation role creation failed for bindingId ########-####-####-####-###########: 1/1 foundations failed. Please check and ensure Platform Services tile is at latest version. The minimum version is 10.4

You might also notice the below set of errors in the Hub UI

2026-05-29T08:40:37.836Z[INFO]Resolving foundations for Org Group 'orggroup-test' (vrn/provider:Tanzu/OrganizationGroup:########-####-####-####-###########)
2026-05-29T08:40:38.389Z[INFO]Resolved 1 foundation(s) for Org Group 'orggroup-test': [#############]
2026-05-29T08:40:38.920Z[INFO]Creating organizations on 1 foundation(s), EDS entities, and role bindings
2026-05-29T08:40:39.878Z[ERROR]Org Group creation failed for 'orggroup-test': Failed to create organization 'orggroup-test' in all 1 foundations. Errors: #############: [Unable to reach the foundation to create org 'orggroup-test', endpointId: ########-####-####-####-###########.]

Under the Tasks you will see Synchronizing Group Role Bindings tasks in failed state and notice the below error:

2026-05-29T08:39:51.435Z[INFO]Resolving foundations for Org Group 'orggroup-test' (vrn/provider:Tanzu/OrganizationGroup:########-####-####-####-###########)
2026-05-29T08:39:51.977Z[INFO]Resolved 1 foundation(s) for Org Group 'orggroup-test': [########-####-####-####-###########]
2026-05-29T08:39:52.506Z[INFO]Creating organizations on 1 foundation(s), EDS entities, and role bindings
2026-05-29T08:40:34.533Z[ERROR]Org Group creation failed for 'orggroup-test': Step 'creating role bindings' failed: Error occurred during role binding creation. Please check the role binding screen for details.

In the graphql-rest-provider-service pod log you will see the below error:

09:39:54.795Z [thread='reactor-http-epoll-2' user='' org='' trace=''] WARN com.vmware.ensemble.graphql.foundation.client.FoundationGraphQLClient - Error while parsing GraphQL errors from JSON response
09:40:13.864Z [thread='reactor-http-epoll-3' user='' org='' trace=''] WARN com.vmware.ensemble.graphql.foundation.client.FoundationGraphQLClient - Error while parsing GraphQL errors from JSON response
09:40:24.200Z [thread='reactor-http-epoll-3' user='' org='' trace=''] WARN com.vmware.ensemble.graphql.foundation.client.FoundationGraphQLClient - Error while parsing GraphQL errors from JSON response

Environment

Tanzu Hub

Cause

This can happen if the Hub is configured with some IDP like LDAP/OIDC/SAML and the corresponding EAR UAA is authenticated with one of those IDPs and the shadow user is not enabled in the EAR tile.

Resolution

While the errors mentioned above doesn't reveal much we can enable the DEBUG logging level in the Platform Services Tile as described here.

Upon enabling the DEBUG logging level in the Platform Services tile, in the ensemble-stitching VM we can observe the below errors:

{"requestId":"########-####-####-####-###########","subgraph":"cloudController","subgraphExecuteId":"########-####-####-####-###########","level":"debug","time":"2026-06-03T09:27:33.383Z","name":"subgraph-execute-end","errors":[{"message":"Upstream HTTP Error: 422, Could not invoke operation POST /v3/roles","path":["encapsulatedCloudController_createRole"],"extensions":{"code":"DOWNSTREAM_SERVICE_ERROR","serviceName":"cloudController","request":{"url":"https://########-####-####-####-###########/v3/roles","method":"POST"},"response":{"status":422,"statusText":"Unprocessable Entity","headers":{"content-length":"140","content-type":"application/json; charset=utf-8","date":"Wed, 03 Jun 2026 09:27:33 GMT","referrer-policy":"strict-origin-when-cross-origin","vary":"Accept, *x-b3-spanid","b3-spanid":"7c5e852584074aec","x-b3-traceid":"0d2bec3d5c80465e7c5e852584074aec","x-content-type-options":"nosniff","x-download-options":"noopen","x-frame-options":"SAMEORIGIN","x-permitted-cross-domain-policies":"none","x-runtime":"0.019016","x-vcap-request-id":"0d2bec3d-5c80-465e-7c5e-852584074aec","x-xsrf-token":"cd286209-d796-4b4d-abb6-487cd4f14f0f","x-xss-protection":"1; mode=block","set-cookie":["NSC_VT-DIE1-OQ2-1ED-DPN-443-WJQ=ffffffff09bf36e845525d5f4f58455e445a4a42378b;path=/;secure;httponly"]},"body":"{\"errors\":[{\"detail\":\"No user exists with the username 'admin@test.org' and origin 'saml'.\",\"title\":\"CF-UnprocessableEntity\",\"code\":10008}]}"}}],"data":{"encapsulatedCloudController_createRole":null}}

The logs clearly show that the user admin@test.org is not present in SAML which is expected. To resolve this we must ensure the "Enable Admins and Org Managers to create UAA shadow users" option is enabled in EAR Tile as documented here.

We need to create only one user with tanzu_platform_admin username and admin@test.org as the email and giving cloud_controller.admin scope in CF UAA, this simplifies the ask to create two users in CF UAA.