Subscribing vCenter Server to VMware cloud fails with error: "Permission to perform this operation was denied."

Products

• VMware

Issue/Introduction

Symptoms:

We are unable to subscribe vCenter Server to cloud console.

 

Error: A general system error occurred subscribing VC: VC_FQDN

 

/var/log/vmware/aap/agents/multi-vc-context-vsphere-entitlement-agent-5a4d5e1e1912:

 

[2023-05-17 07:33:10.008 GMT] [vsphere-entitlement-agent] [vcId='813b9995-429b-4bb0-8dfe-79dd07ab3267' imageId='5a4d5e1e1912' dockerHost='fcc25d125b2b'] [priority='ERROR' thread='scheduled-task-thread24' trace='00000000-0000-0000-8838-c9

e96eba5314'] com.vmware.vsphere.cloud.entitlement.scheduling.VcAgentScheduler@286 - Unexpected error occurred in agent task 'pollForSubscriptionTasks' for VC '813b9995-429b-4bb0-8dfe-79dd07ab3267' in 490 milliseconds.

java.lang.IllegalStateException: There are failed VC subscription tasks

Caused by: com.vmware.vsphere.cloud.entitlement.subscription.SubscriptionConfigurationUpdateException: Assignment.update_Task() failed with VAPI Error: 

Caused by: com.vmware.vapi.std.errors.Unauthorized: Unauthorized (com.vmware.vapi.std.errors.unauthorized) => {

  messages = [LocalizableMessage (com.vmware.vapi.std.localizable_message) => {

  id = com.vmware.vapi.authorization.permission.denied,

  defaultMessage = Permission to perform this operation was denied.,

  args = [],

  params = <null>,

  localized = <null>

}],

  data = <null>,

  errorType = UNAUTHORIZED

 

VC log: /var/log/vmware/sso/ssoAdminServer.log

 

2023-05-17T07:37:11.054Z INFO ssoAdminServer[109:pool-2-thread-10] [OpId=f59e0e3c-9fcf-4224-8145-6fbea5f4b081] [com.vmware.identity.vlsi.RoleBasedAuthorizer] User {Name: machine-8624ee33-4194-47b9-8b42-5f9fb8b99d74, Domain: vsphere.local} with role 'Administrator' is authorized formethod call 'PrincipalDiscoveryService.findNestedParentGroups'

2023-05-17T07:37:11.054Z INFO ssoAdminServer[100:pool-2-thread-3] [OpId=f59e0e3c-9fcf-4224-8145-6fbea5f4b081] [com.vmware.identity.admin.vlsi.PrincipalDiscoveryServiceImpl] [User {Name: machine-8624ee33-4194-47b9-8b42-5f9fb8b99d74, Domain: vsphere.local} with role 'Administrator'] Find nested parent groups foruser {Name: CloudServicesGateway_license-service-admin_1edefa3c-526b-67bf-a78f-81ccad127c69_813b9995-429b-4bb0-8dfe-79dd07ab3267, Domain: VSPHERE.LOCAL}

2023-05-17T07:37:11.057Z INFO ssoAdminServer[100:pool-2-thread-3] [OpId=f59e0e3c-9fcf-4224-8145-6fbea5f4b081] [com.vmware.identity.admin.vlsi.PrincipalDiscoveryServiceImpl] Vmodl method PrincipalDiscoveryService.findNestedParentGroups returnvalue is [(sso.admin.Group) {\n  dynamicType = null,\n  dynamicProperty = null,\n  id = (sso.PrincipalId) {\n   dynamicType = null,\n   dynamicProperty = null,\n   name = Everyone,\n   domain = vsphere.local\n  },\n  alias = null,\n  details = (sso.admin.GroupDetails) {\n   dynamicType = null,\n   dynamicProperty = null,\n   description = \n  }\n}] 

 

 

SSO admin server logs show that the GW license service admin user for vCenter Server (VC ID: 813b9995-429b-4bb0-8dfe-79dd07ab3267) is not part of the License Service Administrators group (LicenseService.Administrators)

Environment

VMware vSphere+ Cloud Services

Resolution

VMware Engineering is working on this issue to provide further updates. 

Workaround:

We need to add "CloudServicesGateway_license-service-admin_1edefa3c-526b-67bf-a78f-81ccad127c69_813b9995-429b-4bb0-8dfe-79dd07ab3267" user back to the License Service Administrators group on the VC.

Steps:
1. Login to vCenter Server.
2. Navigate to : Administration > Users and Groups > Groups
3. Select License Service Administrators Group and add user: CloudServicesGateway_license-service-admin_1edefa3c-526b-67bf-a78f-81ccad127c69_813b9995-429b-4bb0-8dfe-79dd07ab3267.
4. Retry the subscription.

Additional Information

Impact/Risks:

vCenter Server subscription to vSphere+ fails.