Tag assignment task on a virtual machine fails with the error message “vmodl.fault.SystemError.”

Products

• VMware vCenter Server

Issue/Introduction

• Attempting to assign tags to a virtual machine fails with error "vmodl.fault.SystemError"
  
  
• Log file /var/log/vmware/vsphere-ui/logs/vsphere_client_virgo.log contains entries similar to the following: 
  
  [YYYY-MM-DDThh:mm:ss] [WARN ] -nio-127.0.0.1-5090-exec-945 70071483 106548 200780 com.vmware.vim.vmomi.client.http.impl.HttpProtocolBindingBase     Asynchronous execution requested but no Executor configured. The request will be executed as synchronous one.
[YYYY-MM-DDThh:mm:ss] [ERROR] -nio-127.0.0.1-5090-exec-945 70071483 106548 200780 com.vmware.vsphere.client.tagging.impl.TagMutationProvider        An error occurred while attaching tags java.util.concurrent.ExecutionException: (vmodl.fault.SystemError) {
   faultCause = null,
   faultMessage = null,
   reason = Invalid fault
}
..
Caused by: com.vmware.vim.binding.vmodl.fault.SystemError: com.vmware.vapi.std.errors.InternalServerError
..
        at com.sun.proxy.$Proxy1086.bulkAttachTags(Unknown Source)
        at com.vmware.vsphere.client.tagging.impl.TagServiceImpl.attachTags(TagServiceImpl.java:95)
  
  
• Log file /var/log/vmware/vpxd/vpxd.log contains entries similar to the following:
  
YYYY-MM-DDThh:mm:ss error vpxd[123456] [Originator@1234 sub=Tagging opID=123456-1234-1234-1234-123456-23] PolicyCache not initialized
  
  
• Log file /var/log/vmware/vpxd-svcs/vpxd-svcs.log contains entries similar to the following when trace logging is enabled (Enabling debug logging for SSO and Authentication services in vCenter Server):
  
  YYYY-MM-DDThh:mm:ss [correlation-dispatcher-4 [] DEBUG com.vmware.vim.vlsiextensions.ISBodyHandlerImpl  opId=] Service Type: InventoryServiceTagManager, Method: bulkAttachTags
YYYY-MM-DDThh:mm:ss [dataservice-7 [] DEBUG com.vmware.cis.core.tagging.internal.impl.TagAssociationManagerImpl  opId=0a14c34e-2729-4672-ad1e-b76d31bc2f0c IS] Attaching tag urn:vmomi:InventoryServiceTag:39a259ba-b04e-4915-b819-6eb07b56f8fe:GLOBAL to object urn:vmomi:VirtualMachine:vm-98157:7a5b2941-cb1e-4530-b5bf-e3b9d80a7f94 start
YYYY-MM-DDThh:mm:ss [dataservice-7 [] DEBUG com.vmware.cis.core.tagging.internal.helper.AttachLockPool  opId=0a14c34e-2729-4672-ad1e-b76d31bc2f0c IS] Getting lock for object `urn:vmomi:VirtualMachine:vm-98157:7a5b2941-cb1e-4530-b5bf-e3b9d80a7f94` with hashcode `-1798747600`.
YYYY-MM-DDThh:mm:ss [dataservice-7 [] DEBUG com.vmware.cis.core.tagging.internal.impl.TagAssociationManagerImpl  opId=0a14c34e-2729-4672-ad1e-b76d31bc2f0c IS] In listAttachedTagsInt method. TaggingUricom.vmware.cis.server.util.VmomiUri@3a411bb2
YYYY-MM-DDThh:mm:ss [tomcat-exec-58 [] DEBUG com.vmware.vapi.internal.util.AuthenticationUtil  opId=] User name VSPHERE.LOCAL\vpxd-svc-acct-<VC-GUID>
YYYY-MM-DDThh:mm:ss [tomcat-exec-58 [] INFO  com.vmware.vcenter.compute.AuthzPermissionValidator  opId=] Checking privileges for user: VSPHERE.LOCAL\vpxd-svc-acct-<VC-GUID>
YYYY-MM-DDThh:mm:ss [tomcat-exec-58 [] DEBUG com.vmware.cis.core.authz.accesscontrol.impl.CheckPrivilegesRouterRiseImpl  opId=] Call isAllowed for urn:acl:global:permissions
YYYY-MM-DDThh:mm:ss [tomcat-exec-58 [] TRACE com.vmware.cis.authorization.impl.provider.InMemoryAuthCache  opId=] Object not found or deleted: urn:acl:global:permissions
YYYY-MM-DDThh:mm:ss [tomcat-exec-58 [] TRACE com.vmware.cis.authorization.impl.AuthorizationManagerImpl  opId=] Object urn:acl:global:permissions has parent null
YYYY-MM-DDThh:mm:ss [tomcat-exec-58 [] DEBUG com.vmware.vapi.authz.impl.AuthorizationFilter  opId=] Insufficient privileges. Contact the Administrator to get the required privileges.

Environment

vCenter Server 7.x
vCenter Server 8.x

Cause

The issue occurs due to missing permissions for service account vpxd-svc-acct at the vCenter global level.

Resolution

In vSphere Client --> Administration --> Global Permissions, verify that vpxd-svc-acct-xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx account has Administrator role with Propagate to children option.

Using vCenter Server Global Permissions