User wcp-appplatform-user account password unlock and reset procedure
search cancel

User wcp-appplatform-user account password unlock and reset procedure

book

Article ID: 389910

calendar_today

Updated On:

Products

Tanzu Kubernetes Runtime vSphere with Tanzu

Issue/Introduction

  • This article is intended to provide steps to review app-platform user account password sync, provide a workaround to unlock the account, and reset the password manually if unlocking doesn't resolve and passwords are out of sync.

  • This account expiring can cause issues with supervisor upgrades. You might see issues with TKG plugin
    • Service: tkg.vsphere.vmware.com. Reason: ReconcileFailed. Message: kapp: Error: waiting on reconcile packageinstall/tkg-controller (packaging.carvel.dev/v1alpha1) namespace: svc-tkg-domain-c<domain id>:
        Finished unsuccessfully (Reconcile failed:  (message: kapp: Error: waiting on reconcile deployment/tkgs-plugin-server (apps/v1) namespace: svc-tkg-domain-c<domain id>:
        Finished unsuccessfully (Deployment is not progressing: ProgressDeadlineExceeded (message: ReplicaSet "tkgs-plugin-server-7c**" has timed out progressing.)))).
      Service: velero.vsphere.vmware.com. Status: Running

  • From the Supervisor check the failed pod logs using below command

kubectl logs -n svc-tkg-domain-c<domain id>  tkgs-plugin-server-7c**

0304 HH:MM:SS.496086 1 deleg.go:130] vcuiplugin "msg"="Failed to call signcsr" "params"={"caFilename":"cert-tkgs-plugin-ca.cert","csrFilename":"cert-tkgs-plugin.csr","durationDays":"365","outFilename":"cert-tkgs-plugin.cert"} "stdout"="Traceback (most recent call last):\n File \"//pyoperator/remediateNamespace.py\", line 168, in \####\###\n fp.write(vmcaRootCert(vcHost, vcPort))\n File \"//pyoperator/remediateNamespace.py\", line 44, in vmcaRootCert\n with vapihelper.SessionManager(vcHost, vcPort) as session:\n File \"/pyoperator/vapihelper.py\", line 70, in __enter__\n self.stub_cfg, self.session_id = connectVapi(self.vcHost, self.vcPort)\n File \"/pyoperator/vapihelper.py\", line 46, in connectVapi\n session_id = session_svc.create()\n File \"/pyoperator/lib/vapi_common_client-2.100.0-py2.py3-none-any.whl/com/vmware/cis_client.py\", line 205, in create\n File \"/pyoperator/lib/vapi_runtime-2.100.0-py2.py3-none-any.whl/vmware/vapi/bindings/stub.py\", line 345, in _invoke\n File \"/pyoperator/lib/vapi_runtime-2.100.0-py2.py3-none-any.whl/vmware/vapi/bindings/stub.py\", line 295, in native_invoke\ncom.vmware.vapi.std.errors_client.Unauthenticated: {challenge : SIGN realm=\"<>\",sts=\"https://<vCenter>/sts/STSService/tanzu.local\", Basic realm=\"vCenter\", messages : [], data : None, error_type : UNAUTHENTICATED}\n"

  • From vCenter Server, refer the endpoint.log under location /var/log/vmware/vapi/endpoint/endpoint.log.*


    2025-02-25T03:00:00.245Z | WARN | vAPI-I/O dispatcher-0 | RemoteSessionService | ebdb7ddf-6972-49ab-acde-83f643249182 | Received an error while notifying for session activity
    com.vmware.vapi.std.errors.Unauthenticated: Unauthenticated (com.vmware.vapi.std.errors.unauthenticated) => {
        messages = [],
        data = <null>,
        errorType = UNAUTHENTICATED,
        challenge = SIGN realm="<>",sts="https://<vCenter fqdn>/sts/STSService/<domain>.local", Basic realm="vCenter"
    }
     at com.vmware.vapi.std.errors.Unauthenticated._newInstance2(Unauthenticated.java:241) ~[vapi-runtime.jar:?]
     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_401]
     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_401]
     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_401]

  • From VC, check the vmdird logs under location /var/log/vmware/vmdird

2025-02-28T13:32:47.013Z:t@139601335252544:ERROR: Bind Request Failed (127.0.0.1) error 49: Protocol version: 3, Bind DN: "CN=wcp-appplatform-user-domain-c<domain>-<id>,cn=ServicePrincipals,dc=<domain>,dc=local", Method: SASL
2025-02-28T13:37:32.553Z:t@139601326859840:ERROR: SASLSessionStep: sasl error (-13)(SASL(-13): authentication failure: client evidence does not match what we calculated. Probably a password error)

Environment

vSphere Supervisor 8.0

Cause

Password mismatch due to failed account refresh by wcp.

Resolution

To resolve the password mismatch between the Supervisor and vCenter Server, you must manually synchronize the appplatform-svcacct password. Follow the steps below to extract the correct password from the vCenter database and update the Supervisor secret.

1. Extract the Current Service Account Password

  • Take a SSH/Putty session to the vCenter Server

  • Create a custom script by copying the existing decryption utility:
    • cp /usr/lib/vmware-wcp/decryptK8Pwd.py /usr/lib/vmware-wcp/appplatform_svcacct.py

  • Modify the script to target the appplatform_svcacct_pwd field:
    • vi /usr/lib/vmware-wcp/appplatform_svcacct.py
    • Change lines

From
   cur = conn.cursor()
    query = "select cluster, master_mgmt_ip, password"
   .. 
        pt = decrypt(row[2], key)
        print("Cluster: %s" % row[0])
        print("IP: %s" % row[1])
        print("PWD: %s" % pt)
        if args.show_grubpwd:
            print("GRUB_PWD: %s" 
 print("-" * 60 + "\n")        
   
To:    
 cur = conn.cursor()
    query = "select cluster, appplatform_svcacct_pwd"
    ...
        pt = decrypt(row[1], key)
        print("Cluster: %s" % row[0])
        print("appplatform_svcacct_pwd: %s" % pt)
        print("-" * 60 + "\n")

  • Change the permission for the file 
    • chmod +x /usr/lib/vmware-wcp/appplatform_svcacct.py

  • Run the file
    • /usr/lib/vmware-wcp/appplatform_svcacct.py

Output:
Cluster: domain-c<domain>:a9082462-0edf-4222-aeb9-ae18642e5082
Appplatform_svcacct_pwd: _w##########

2. Update the Supervisor Secret

  • Take SSH/Putty Session to the Supervisor or Jumpbox with Supervisor Context.
  • Verify the current secret to confirm a mismatch:

root@<> [ ~ ]# k get secret -n vmware-system-appplatform-operator-system vmware-system-appplatform-vc-auth -o jsonpath='{.data.password}' |base64 -d; echo
ab########

  • Encode the password retrieved from VC using the /usr/lib/vmware-wcp/appplatform_svcacct.py command:
     
    # echo -n '_wT#######' | base64
  • Update the secret, edit the secret and update encoded password 

 kubectl edit secret -n vmware-system-appplatform-operator-system vmware-system-appplatform-vc-auth

  • Restart app-platform pod and any failed pods such as tkgs-plugin.

kubectl delete pod -n vmware-system-appplatform-operator-system vmware-system-appplatform-operator-mgr-0
kubectl delete pod -n svc-tkg-domain-c<> tkgs-plugin-server-<>

    • Note: Ensure you replace <domain_id> and <tkgs-plugin-server-pod-name> with the specific identifiers found in your environment.

3. Restart Services on vCenter Server

  • From vCenter restart wcp

service-control --restart wcp

 



Powered by Wolken Software