Content Library sync fails with 'Access to perform the operation was denied' or 'INVALID_ELEMENT_TYPE'

Products

• VMware vCenter Server

Issue/Introduction

When attempting to synchronize a Subscribed Content Library with a Publisher vCenter, the operation fails with one or more of the following symptoms:

• Error in vSphere Client: "Access to perform the operation was denied."
• Error in cls.log: com.vmware.vapi.std.errors.Unauthenticated: {messages [com.vmware.vapi.endpoint.method.authentication.required]}
• Error in cls.log: INVALID_ELEMENT_TYPE reported during the metadata fetch.
• Manual synchronization attempts fail immediately.

Environment

• VMware vCenter Server 8.0

Cause

The issue is due to multiple STS certificates stored in the vmdir causing the content-library service to validate the token with the wrong STS certificate.

Specifically, using the vCert tool reveals multiple TenantCredential entries under STS Signing Certs and Signing chains. This typically occurs when previous STS certificates were not properly cleaned up during a renewal or after a vCenter restoration from backup. The Content Library service attempts to authenticate using an expired or stale credential from the database, leading to the UNAUTHENTICATED error.

Resolution

To resolve this, the duplicate and stale STS credentials must be consolidated by replacing the STS signing certificate using the vCert utility.

1. Download and Install vCert: Ensure the latest version of the vCert tool is available on the vCenter Server Appliance.

2. Critical: Before proceeding, you must take offline snapshots of ALL vCenter Server and PSC nodes within the SSO domain (ELM group). Modifying the STS identity store can render the environment inoperable if interrupted.

3. Launch vCert: Run the tool via SSH:

   ./vCert.py

4. Navigate to Management:
   • Select Option 3: Manage Certificates.
   • Select Option 8: STS signing certificates.
5. Execute Replacement:
   • Select Option 1: Replace STS Signing certificate with a VMCA-signed certificate.
   • Provide the local SSO administrator credentials ([e-mail]) when prompted.
6. Verify Cleanup: vCert will generate a new STS certificate and automatically remove the stale TenantCredential entries. Verify that only one valid entry remains.
7. Restart Services: Restart all services on all vCenter nodes in the Enhanced Linked Mode (ELM) group to apply the new certificate:

   service-control --stop --all && service-control --start --all

8. Validate Sync: Log in to the vSphere Client and initiate a manual sync of the Content Library. The status should now transition to "Succeeded."