User wcp-appplatform-user account password unlock and reset procedure
Article ID: 389910
Updated On:
Products
Issue/Introduction
This article is intended to provide steps to review app-platform user account password sync, provide a workaround to unlock the account, and reset the password manually if unlocking doesn't resolve and passwords are out of sync.
This account expiring can cause issues with supervisor upgrades. You might see issues with TKG plugin
• Service: tkg.vsphere.vmware.com. Reason: ReconcileFailed. Message: kapp: Error: waiting on reconcile packageinstall/tkg-controller (packaging.carvel.dev/v1alpha1) namespace: svc-tkg-domain-c<domain id>: Finished unsuccessfully (Reconcile failed: (message: kapp: Error: waiting on reconcile deployment/tkgs-plugin-server (apps/v1) namespace: svc-tkg-domain-c<domain id>: Finished unsuccessfully (Deployment is not progressing: ProgressDeadlineExceeded (message: ReplicaSet "tkgs-plugin-server-7c**" has timed out progressing.)))). • Service: velero.vsphere.vmware.com. Status: Running
In logs for failed pod
kubectl logs -n svc-tkg-domain-c<domain id> tkgs-plugin-server-7c**
0304 09:16:41.496086 1 deleg.go:130] vcuiplugin "msg"="Failed to call signcsr" "params"={"caFilename":"cert-tkgs-plugin-ca.cert","csrFilename":"cert-tkgs-plugin.csr","durationDays":"365","outFilename":"cert-tkgs-plugin.cert"} "stdout"="Traceback (most recent call last):\n File \"//pyoperator/remediateNamespace.py\", line 168, in \u003cmodule\u003e\n fp.write(vmcaRootCert(vcHost, vcPort))\n File \"//pyoperator/remediateNamespace.py\", line 44, in vmcaRootCert\n with vapihelper.SessionManager(vcHost, vcPort) as session:\n File \"/pyoperator/vapihelper.py\", line 70, in __enter__\n self.stub_cfg, self.session_id = connectVapi(self.vcHost, self.vcPort)\n File \"/pyoperator/vapihelper.py\", line 46, in connectVapi\n session_id = session_svc.create()\n File \"/pyoperator/lib/vapi_common_client-2.100.0-py2.py3-none-any.whl/com/vmware/cis_client.py\", line 205, in create\n File \"/pyoperator/lib/vapi_runtime-2.100.0-py2.py3-none-any.whl/vmware/vapi/bindings/stub.py\", line 345, in _invoke\n File \"/pyoperator/lib/vapi_runtime-2.100.0-py2.py3-none-any.whl/vmware/vapi/bindings/stub.py\", line 295, in native_invoke\ncom.vmware.vapi.std.errors_client.Unauthenticated: {challenge : SIGN realm=\"<>\",sts=\"https://<vCenter>/sts/STSService/tanzu.local\", Basic realm=\"vCenter\", messages : [], data : None, error_type : UNAUTHENTICATED}\n"
From /var/log/vmware/vapi/endpoint/endpoint.log.*
2025-02-25T03:00:00.245Z | WARN | vAPI-I/O dispatcher-0 | RemoteSessionService | ebdb7ddf-6972-49ab-acde-83f643249182 | Received an error while notifying for session activity
com.vmware.vapi.std.errors.Unauthenticated: Unauthenticated (com.vmware.vapi.std.errors.unauthenticated) => {
messages = [],
data = <null>,
errorType = UNAUTHENTICATED,
challenge = SIGN realm="<>",sts="https://<vCenter fqdn>/sts/STSService/<domain>.local", Basic realm="vCenter"
}
at com.vmware.vapi.std.errors.Unauthenticated._newInstance2(Unauthenticated.java:241) ~[vapi-runtime.jar:?]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_401]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_401]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_401]
.....
Checking /var/log/vmware/vmdird
2025-02-28T13:32:47.013Z:t@139601335252544:ERROR: Bind Request Failed (127.0.0.1) error 49: Protocol version: 3, Bind DN: "CN=wcp-appplatform-user-domain-c<domain>-<id>,cn=ServicePrincipals,dc=<domain>,dc=local", Method: SASL2025-02-28T13:37:32.553Z:t@139601326859840:ERROR: SASLSessionStep: sasl error (-13)(SASL(-13): authentication failure: client evidence does not match what we calculated. Probably a password error)
Environment
vSphere Supervisor 8.0
Cause
Password mismatch due to to failed account refresh by wcp.
Resolution
Complete following steps to reset password.
cp /usr/lib/vmware-wcp/decryptK8Pwd.py /usr/lib/vmware-wcp/appplatform_svcacct.py
vi usr/lib/vmware-wcp/appplatform_svcacct.py
Change lines
From cur = conn.cursor() query = "select cluster, master_mgmt_ip, password" .. pt = decrypt(row[2], key) print("Cluster: %s" % row[0]) print("IP: %s" % row[1]) print("PWD: %s" % pt) if args.show_grubpwd: print("GRUB_PWD: %s" print("-" * 60 + "\n") To: cur = conn.cursor() query = "select cluster, appplatform_svcacct_pwd" ... pt = decrypt(row[1], key) print("Cluster: %s" % row[0]) print("appplatform_svcacct_pwd: %s" % pt) print("-" * 60 + "\n")
chmod +x /usr/lib/vmware-wcp/appplatform_svcacct.py
Run the file
/usr/lib/vmware-wcp/appplatform_svcacct.py
Cluster: domain-c<domain>:a9082462-0edf-4222-aeb9-ae18642e5082Appplatform_svcacct_pwd: _wT>F.',~|5"7}Z}{l/{
------------------------------------------------------------
Compare to secret if they are different.
root@<> [ ~ ]# k get secret -n vmware-system-appplatform-operator-system vmware-system-appplatform-vc-auth -o jsonpath='{.data.password}' |base64 -d; echo@)T5erI8j"~mM{LFd&oX
From supervisor Node SSH, use base64 to encode the password output from the /appplatform_svcacct.py command:
# echo -n '_wT>F.',~|5"7}Z}{l/{' | base64
Edit the secret and update password value to encoded step from above.
kubectl edit secret -n vmware-system-appplatform-operator-system vmware-system-appplatform-vc-auth
Restart app platform pod and any failed pods such as tkgs-plugin.
kubectl delete pod -n vmware-system-appplatform-operator-system vmware-system-appplatform-operator-mgr-0
k delete pod -n svc-tkg-domain-c<> tkgs-plugin-server-<>
From vCenter restart wcp
service-control --restart wcp
Feedback
