vSphere with Tanzu Deployment Fails very early on trying to create a resource pool due to a permissions issue.
search cancel

vSphere with Tanzu Deployment Fails very early on trying to create a resource pool due to a permissions issue.

book

Article ID: 323429

calendar_today

Updated On:

Products

VMware vCenter Server

Issue/Introduction

Symptoms:
Deploy of vSphere with Tanzu fails almost immediately. 
Create Resource Pool and delete Resource Pool tasks fail are spammed in the vCenter's task and events bar.
In the wcpsvc.log you will see something similar to
  
2021-04-06T19:59:38.453Z warning wcp [opID=60504f33-domain-c9] Unable to create root entities in vCenter for domain-c9. Err soap.soapFaultError:ServerFaultCode: The user or group named 'vpxd-extension-2xxxxx57-9e53-4d49-a8b0-633xxxxxdc2' does not exist.
 
2021-04-06T19:59:50.248Z error wcp [opID=6051321c-domain-c9] Could not set read-only permissions on entity ResourcePool:resgroup-12066.
 
*The previous log messages are examples and will vary between environments


Environment

VMware vCenter Server 7.0.x

Cause

This is due to a missing localos identity source. You can validate that the identity source is missing by running… 
root@vcsa-01a [ ~ ]# sso-config.sh -get_identity_sources
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/vmware/lib64/log4j-slf4j-impl-2.11.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/vmware-sso/vmware-sts/webapps/ROOT/WEB-INF/lib/log4j-slf4j-impl-2.11.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]

Total number of identitysources retrieved for tenant:vsphere.local : 3
(If the value is undefined against a param, then you might notice "UndefinedConfig" against it.)

********** IDENTITY SOURCE INFORMATION **********
IdentitySourceName        :  vsphere.local
DomainType                :  SYSTEM_DOMAIN

********** IDENTITY SOURCE INFORMATION **********
IdentitySourceName        :  corp.local
DomainType                :  EXTERNAL_DOMAIN
Identity Settings:
  alias                   :  CORP
  authenticationType      :  USE_KERBEROS
  userBaseDN              :  dc=corp,dc=local
  groupBaseDN             :  dc=corp,dc=local
  username                :  UndefinedConfig
  providerType            :  IDENTITY_STORE_TYPE_ACTIVE_DIRECTORY
  servicePrincipalName    :  UndefinedConfig
  useMachineAccount       :  true
  FriendlyName            :  CORP
  SearchTimeoutInSeconds  :  300
Connection Settings:
URLs:
    0:  ldap://corp.local
Certificates:
Attributes:
  http://schemas.xmlsoap.org/claims/UPN                           :  userPrincipalName
  http://rsa.com/schemas/attr-names/2009/01/GroupIdentity         :  memberof
  http://schemas.xmlsoap.org/ws/2005/05/identity/claims/givenname :  givenName
  http://schemas.xmlsoap.org/ws/2005/05/identity/claims/surname   :  sn
  http://vmware.com/schemas/attr-names/2011/07/isSolution         :  subjectType
Flags::
  Flags=0; [Default: recursively computing nested groups, no site affinity is enabled for AD over Ldap identity providers.]
root@vcsa-01a [ ~ ]#

Expected output would be:  
root@vcsa-01a [ ~ ]# sso-config.sh -get_identity_sources
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/vmware/lib64/log4j-slf4j-impl-2.11.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/vmware-sso/vmware-sts/webapps/ROOT/WEB-INF/lib/log4j-slf4j-impl-2.11.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]

Total number of identitysources retrieved for tenant:vsphere.local : 3
(If the value is undefined against a param, then you might notice "UndefinedConfig" against it.)

********** IDENTITY SOURCE INFORMATION **********
IdentitySourceName        :  vsphere.local
DomainType                :  SYSTEM_DOMAIN

********** IDENTITY SOURCE INFORMATION **********
IdentitySourceName        :  localos
DomainType                :  LOCAL_OS_DOMAIN

********** IDENTITY SOURCE INFORMATION **********
IdentitySourceName        :  corp.local
DomainType                :  EXTERNAL_DOMAIN
Identity Settings:
  alias                   :  CORP
  authenticationType      :  USE_KERBEROS
  userBaseDN              :  dc=corp,dc=local
  groupBaseDN             :  dc=corp,dc=local
  username                :  UndefinedConfig
  providerType            :  IDENTITY_STORE_TYPE_ACTIVE_DIRECTORY
  servicePrincipalName    :  UndefinedConfig
  useMachineAccount       :  true
  FriendlyName            :  CORP
  SearchTimeoutInSeconds  :  300
Connection Settings:
URLs:
    0:  ldap://corp.local
Certificates:
Attributes:
  http://schemas.xmlsoap.org/claims/UPN                           :  userPrincipalName
  http://rsa.com/schemas/attr-names/2009/01/GroupIdentity         :  memberof
  http://schemas.xmlsoap.org/ws/2005/05/identity/claims/givenname :  givenName
  http://schemas.xmlsoap.org/ws/2005/05/identity/claims/surname   :  sn
  http://vmware.com/schemas/attr-names/2011/07/isSolution         :  subjectType
Flags::
  Flags=0; [Default: recursively computing nested groups, no site affinity is enabled for AD over Ldap identity providers.]
root@vcsa-01a [ ~ ]#

Resolution

1. run the following commands to stop vpxd services:

vmon-cli -k vpxd
vmon-cli -k vpxd-svcs

2. run command below to create the localos identity source:

sso-config.sh -add_identity_source -type 'localos' -i 'localos'

3. run command below to confirm localos identity source is created:

sso-config.sh -get_identity_sources

The output should include the following:

********** IDENTITY SOURCE INFORMATION **********

IdentitySourceName        :  localos
DomainType                :  LOCAL_OS_DOMAIN

4. run the following commands to start vpxd services:

vmon-cli -i vpxd
vmon-cli -i vpxd-svcs


Then retry the workload management enablement.

Additional Information

Impact/Risks:
Make sure to take powered off snapshots of all vCenter's before running this as it will make edits to the vmdir database. If you have any issues after the workaround please grab a vCenter log bundle, then revert the vCenters, then open a case with VMware Support.

Powered by Wolken Software