Error: "minorErrorCode":"invalid_grant","message":"Invalid refresh token"" in logs and Encryption Management UI fails to load

Products

• VMware Cloud Director

Issue/Introduction

• The Encryption Management (BYOK) page in the VMware Cloud Director (VCD) UI fails to load or spins indefinitely, blocking access to manage encrypted VMs.
• Virtual Machine creation or customization tasks may fail.
• The Encryption Management appliance service (byok-server.service) fails to start and remains in a crash loop.
• The following error is observed in the appliance logs /opt/vmware/byok/logs/byok-error.log

time="####-##-###########" level=error msg="cloud director endpoint https://<VCD-FQDN>:443/oauth/provider/token failed: {\"error\":\"invalid_grant\",\"error_description\":\"Invalid refresh token\",\"error_uri\":null,\"minorErrorCode\":\"invalid_grant\",\"message\":\"Invalid refresh token\",\"stackTrace\":null}"

Environment

• VMware Cloud Director 10.6.x
• VMware Cloud Director Encryption Management 1.2.1

Cause

This issue can occur when the refresh token stored in the Encryption Management appliance's local configuration (/opt/vmware/byok/etc/config.yaml) no longer matches the token expected by VMware Cloud Director.

Resolution

To resolve this issue in Encryption Management 1.2.1, A clean removal and reinstallation of the add-on configuration should be performed.
This process will force a new OAuth Device Authorization flow and synchronize a fresh token.

Note: Proceeding with a full removal and reinstallation of the add-on will temporarily prevent existing end-tenants from performing encryption-related tasks (such as managing keys or creating encrypted VMs). This reconnection does not happen automatically. Once the provider completes the resolution steps below, tenant administrators must manually log into their VCD Tenant Portal to re-authenticate their sessions (detailed in Step 10).

1. Log into the VCD Provider Portal as a System Administrator.
2. Navigate to Administration > Access Control > Service Accounts and verify the status of the encryption-management-system-user service account should be 'Active'.
3. Ensure the Encryption Management ISO is present in VCD and attached to the Encryption Management appliance. Locate the appliance VM, click All Actions > Media > Insert Media, select the VMware Cloud Director Encryption Management 1.2.1 ISO file, and click Insert.
4. Open an SSH session to the Encryption Management appliance as the root user.
5. Execute the following command to cleanly remove the existing add-on registration:
   
   /mnt/cdrom/vcdemctl addon remove
   
   
6. Once the removal completes successfully, execute the installation command:
   
   /mnt/cdrom/vcdemctl addon install
   
   
7. The installer will pause and provide a User Code on the screen.
8. Return to the VCD Provider Portal, navigate to Administration > Access Control > Service Accounts > Review Access Requests, input the User Code, and click GRANT.
9. The installer will automatically resume, create a new service account, securely encrypt a fresh token, and start the byok-server service.
10. Notify existing end-tenants that they must log into their respective VCD Tenant Portals and navigate to the Encryption Management UI to manually re-authenticate their encryption sessions.