vSphere with Tanzu WCP service authentication failure for wcp-storage-user

Products

• VMware vSphere Kubernetes Service

Issue/Introduction

Issue where the WCP service fails to function correctly, matching the symptoms described in KB 420549. Restarting the WCP service does not resolve the issue. Upon reviewing the wcpsvc.log files, authentication failures associated with the wcp-storage-user account are present.

Environment

VMware vSphere with Tanzu

Cause

The password for the wcp-storage-user account in vSphere Single Sign-On (SSO) does not match the stored password within the vsphere-config-secret Kubernetes secret.

Resolution

To resolve this issue, confirm the stored secret and reset the SSO account password to match:

1. Confirm the contents of the secret to retrieve the username and password.

   • Access the Supervisor server as root and run the following command:

     kubectl get secrets vsphere-config-secret -n vmware-system-csi -o jsonpath='{.data.vsphere-cloud-provider\.conf}' | base64 -d
     

     Example Output:

     user = "wcp-storage-user-97c0374e-[e-mail]"
     password = "VMware123!"
     

2. Confirm that the user ID from the output matches the account experiencing issues in the wcpsvc.log files.

3. In the vCenter command line verify the account state using the dir-cli command to ensure the account is not locked and the password expiry is in the future:

   /usr/lib/vmware-vmafd/bin/dir-cli user find-by-name --account wcp-storage-user-<ID> --level 2
   

   Example Output:

   Enter password for [email protected]:
   Account: wcp-storage-user-97c0374e-4dc8-40f0-820a-9f794ebe7d2e-644b4ae9-6221-4fd3-b042-d12ac7ea6e7f
   UPN: wcp-storage-user-97c0374e-[e-mail]
   Account disabled: FALSE
   Account locked: FALSE
   Password never expires: FALSE
   Password expired: FALSE
   Password expiry: 89 day(s) 23 hour(s) 53 minute(s) 33 second(s)
   

4. Reset the SSO password to match the secret password identified in step 1. Run the following command:

   /usr/lib/vmware-vmafd/bin/dir-cli password reset --account wcp-storage-user-<ID> --new 'DESIRED_PASSWORD'
   

   Example Output:

   Enter password for [email protected]:
   Password was reset successfully for [wcp-storage-user-97c0374e-4dc8-40f0-820a-9f794e

Additional Information

If your actual password contains a single quote ('), change the surrounding single quotes in the dir-cli password reset command to double quotes (") to ensure the command is parsed correctly.