NCP authentication fails when using a complex password. Error: Authentication Failed: Empty Password

Products

• VMware NSX Networking
• VMware NSX
• VMware NSX-T Data Center

Issue/Introduction

Symptoms:

• In a Tanzu deployment using NCP, TAS fails to push applications.
• NCP is configured to authenticate to NSX with username and password (vIDM / LDAP) and the password contains special characters.
• When checking the NCP logs (ncp/ncp.stdout.log) you see similar entries:
  NSX 13680 - [nsx@6876 comp="nsx-container-ncp" subcomp="ncp" level="WARNING"] vmware_nsxlib.v3.cluster Session create failed for endpoint https://<nsx-fqdn-or-ip.domain> with response 403, error message: Authentication Failed: Empty Password
NSX 13680 - [nsx@6876 comp="nsx-container-ncp" subcomp="ncp" level="WARNING"] nsx_ujo.common.controller ActualLRPController worker 5 failed to sync ########## due to nsx manager exception: Unexpected error from backend manager (['<nsx-fqdn-or-ip.domain>']) for get_node_logical_ports: Unable to execute search query for node ############ on NSX backend: The credentials were incorrect or the account specified has been locked.

• From the NSX logs var/log/proxy/reverse-proxy.log, you see similar entries:
  ERROR https-jsse-nio-10.81.1.29-443-exec-3 NsxApiSessionAuthenticationFailureHandler 2280937 - [nsx@6876 comp="nsx-manager" errorCode="MP403" level="ERROR" subcomp="http"] Authentication Failed
2024-09-16T09:12:50.842Z  INFO https-jsse-nio-x.x.x.x-443-exec-24 NsxBasicAuthenticationFilter 2280937 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="http"] Account is temporarily locked
2024-09-16T09:12:50.842Z ERROR https-jsse-nio-x.x.x.x-443-exec-24 NsxRestAuthenticationEntryPoint 2280937 - [nsx@6876 comp="nsx-manager" errorCode="MP403" level="ERROR" subcomp="http"] The credentials were incorrect or the account specified has been locked.

Environment

VMware NSX with NCP

Cause

• If NCP is configured to authenticate to NSX with username and password (vIDM / LDAP) and if the password contains one of those 2 special characters "%" and "&", it won't be escaped properly by NCP, causing NSX authentication to fail.

Resolution

Currently, there's no permanent fix for this issue.

Workaround:

1. Do not use characters "&" and "%" in NSX passwords for NCP authentication.
2. Or use Principal Identities Authentication for the NSX-T, instead of LDAP authentication. (preferred)

Additional Information

This issue is present on all NCP releases up to 4.2.1