vSphere with Tanzu supervisor cluster shows license expired when assigned perpetual license
Article ID: 421296
Updated On:
Products
VMware vSphere Kubernetes Service
Issue/Introduction
A supervisor cluster with a non-expiring license experiences an expired license error in the vCenter GUI.
Supervisor cluster and guest cluster operations experience errors due to the expired license error.
Similar errors to the following are found in the wcpsvc.log
YYYY-MM-DDTHH:MM:SS error wcp [compatibility/setupinfo.go:1332] [opID=licenseRefreshMonitor] Error checking license of hosts [host-##### host-## host-## host-##]: ServerFaultCode: Access to perform the operation was denied.
YYYY-MM-DDTHH:MM:SS debug wcp [compatibility/setupinfo71.go:97] [opID=licenseExpirationRefreshMonitor] Host host-## in cluster ######-### is not licensed for namespaces.
YYYY-MM-DDTHH:MM:SS error wcp [vclib/client.go:1485] [opID=licenseExpirationRefreshMonitor] Error completing req {LicenseAssignmentManager:LicenseAssignmentManager [{host-##### wcp} {host-## wcp} {host-## wcp} {host-## wcp}]}. Error: ServerFaultCode: Access to perform the operation was denied.
vpxd logs show user permissions errors related to License Asset Manager operations
YYYY-MM-DDTHH:MM:SS info vpxd[09493] [Originator@6876 sub=vmomi.soapStub[7] opID=wcp-###############################-##] SOAP request returned HTTP failure; <<cs p:################, TCP:localhost:1080>, /ls/sdk>, method: isFeatureAvailable; code: 500(Internal Server Error); fault: (vmodl.fault.SecurityError) {
--> faultCause = (vmodl.MethodFault) null,
--> faultMessage = <unset>
--> msg = "Received SOAP response fault from [<<cs p:################, TCP:localhost:1080>, /ls/sdk>]: isFeatureAvailable
--> Authorization result: User does not have admin rights to perform the operation (########-####-####-####-############)"
--> }
YYYY-MM-DDTHH:MM:SS warning vpxd[09493] [Originator@6876 sub=Vmomi opID=wcp-###############################-##] VMOMI activation LRO failed; <<########-####-####-####-############, <TCP '127.0.0.1 : 8085'>, <TCP '127.0.0.1 : 53468'>>, LicenseAssignmentManager, vim.LicenseAssignmentManager.isFeatureAvailable, <vim.version.v8_0_3_0, official, 8.0.3.0>, {stm: {<io_obj p:0x00007f802c00b920, h:128, <TCP '127.0.0.1 : 8085'>, <TCP '127.0.0.1 : 53470'>>, id: 38617, state(in/out): 3/1}, session: <########-####-####-####-############, <TCP '127.0.0.1 : 8085'>, <TCP '127.0.0.1 : 53468'>>, req: {POST, /sdk}}>, N5Vmomi5Fault13SecurityError9ExceptionE(Fault cause: vmodl.fault.SecurityError
--> )
--> [context]######################################################################################################################################################==[/context]
YYYY-MM-DDTHH:MM:SS info vpxd[09493] [Originator@6876 sub=vpxLro opID=wcp-###############################-##] [VpxLRO] -- FINISH lro-######
YYYY-MM-DDTHH:MM:SS error vpxd[09493] [Originator@6876 sub=Default opID=wcp-###############################-##] [VpxLRO] -- ERROR lro-###### -- ########-####-####-####-############(########-####-####-####-############) -- LicenseAssignmentManager -- vim.LicenseAssignmentManager.isFeatureAvailable: :vmodl.fault.SecurityError
--> Result:
--> (vmodl.fault.SecurityError) {
--> faultCause = (vmodl.MethodFault) null,
--> faultMessage = <unset>
--> msg = "Received SOAP response fault from [<<cs p:################, TCP:localhost:1080>, /ls/sdk>]: isFeatureAvailable
--> Authorization result: User does not have admin rights to perform the operation (########-####-####-####-############)"
--> }
--> Args:
-->
--> Arg entityFeaturePair:
--> (vim.LicenseAssignmentManager.EntityFeaturePair) [
--> (vim.LicenseAssignmentManager.EntityFeaturePair) {
--> entityId = "host-##",
--> feature = "wcp"
Within cis-license/license.log, :
YYYY-MM-DDTHH:MM:SS ##########-#####-########-# operationID=wcp-###############################-## INFO server.core.oldmanagement.vmomi.LicenseManagerMoImpl getEvaluation
YYYY-MM-DDTHH:MM:SS ##########-#####-########-# operationID=wcp-###############################-## INFO common.vmomi.authn.impl.SsoLoginHelperImpl login: User 'vpxd-########-####-####-####-############@vsphere.local' is already authenticated
YYYY-MM-DDTHH:MM:SS ##########-#####-########-# operationID=wcp-###############################-## INFO server.core.oldmanagement.vmomi.LicenseManagerMoImpl getEvaluation DONE
YYYY-MM-DDTHH:MM:SS #####-########-# WARN common.vmomi.authz.impl.PrivilegeAuthorizerImpl authorize: Authorization result: User does not have admin rights to perform the operation (########-####-####-####-############): sessionNonce: '########-####-####-####-############' sessionUser: 'vpxd-########-####-####-####-############@vsphere.local' requestUri: '/ls/sdk' requestContext: '{realUser=VSPHERE.LOCAL\vpxd-extension-########-####-####-####-############, operationID=wcp-###############################-##}'
When running the following command, the vpxd and vpxd-extension users do no appear in the LicenseService.Administrators group for the vCenter.
ldapsearch -o ldif-wrap=no -LLL -h localhost -b "CN=LicenseService.Administrators,dc=vsphere,dc=local" -s sub -D "cn=Administrator,cn=Users,dc=vsphere,dc=local" -WEnvironment
vSphere with Tanzu
vCenter 8.X
Cause
The vpxd users were removed from the LicenseService.Administrators group
Resolution
These accounts must be added back to the LicenseService.Administrators group for the Supervisor and WCP operations to function.
NOTE: Please make sure proper backups and snapshots of the vCenter are in place before making any changes to the environment.
Update the following command to add the vpxd and vpxd-extension users to the LicenseService.Administrators group using the local administrator credentials.
/usr/lib/vmware-vmafd/bin/dir-cli group modify --name LicenseService.Administrators --add vpxd-###-####-########-####-####-####-############ --login "[email protected]" --password "<localadministratorpassword"
Validate the users were added with the following command:
/usr/lib/vmware-vmafd/bin/dir-cli group list --name LicenseService.Administrators --login "[email protected]" --password "<localadministratorpassword>"
CN=vpxd-###-####-########-####-####-####-############,cn=ServicePrincipals,dc=vsphere,dc=localFeedback
Yes
No
Powered by
