Identity Manager users locked out of NSX-T 4.1.1.x after 15 minutes with "Access Denied – Error 403"

Products

• VMware NSX

Issue/Introduction

• 15 minutes after logging into NSX-T with an Identity Manager user the account gets locked with:

“Access Denied – Error 403 You are no longer authenticated. Please login again”

• You see messages similar to the following in the /var/log/proxy/reverse-proxy.log file on the NSX manager:
  
  

  2023-09-14T06:36:44.421Z ERROR Processing request 5ba3cd65-####-####-####-##########02 SessionInvalidationFilter 2744 - [nsx@6876 comp="nsx-manager" errorCode="MP1" level="ERROR" subcomp="http"] Authentication validation failed.
  org.springframework.security.authentication.BadCredentialsException: Invalid credentials
          at com.vmware.nsx.management.rp.security.oauth2.OAuth2AuthenticationProvider.attemptAuthentication(OAuth2AuthenticationProvider.java:140) ~[libreverse-proxy-compile.jar:?]
          at com.vmware.nsx.management.rp.security.oauth2.OAuth2AuthenticationProvider.authenticate(OAuth2AuthenticationProvider.java:125) ~[libreverse-proxy-compile.jar:?]
          at org.springframework.security.authentication.ProviderManager.authenticate(ProviderManager.java:182) ~[spring-security-core-5.6.9.jar:5.6.9]
          at com.vmware.nsx.management.rp.security.SessionInvalidationFilter.validateSession(SessionInvalidationFilter.java:153) ~[libreverse-proxy-compile.jar:?]
          at com.vmware.nsx.management.rp.security.SessionInvalidationFilter.doFilter(SessionInvalidationFilter.java:97) ~[libreverse-proxy-compile.jar:?]
  ..
  Caused by: org.springframework.security.oauth2.client.resource.OAuth2AccessDeniedException: Access token denied.
  ..
  Caused by: org.springframework.security.oauth2.common.exceptions.InvalidGrantException: Invalid username or password
  ..
  2023-09-14T06:36:44.421Z ERROR Processing request 5ba3cd65-####-####-####-##########02 NsxRestAuthenticationEntryPoint 2744 - [nsx@6876 comp="nsx-manager" errorCode="MP403" level="ERROR" subcomp="http"] The credentials were incorrect or the account specified has been locked.

• In the Identity Manager UI under Dashboard -> Reports -> Audit events you see PASSWORD_LOCKEDOUT failed entries.
• The issue does not affect the NSX-T local admin account
• The issue appears after upgrading NSX-T to version 4.1.1.x or above.

Environment

VMware Identity Manager 3.3.x
VMware NSX 4.1.1

Cause

In NSX 4.1.1 there was a modification made to the NSX OAuth app in VIDM so that NSX won't issue as many refresh tokens.

Resolution

This issue is resolved in VMware NSX 4.1.2.5
This issue is resolved in VMware NSX 4.2.0

Workaround:

1. Login to the Identity manager's administrator console:
   
   https://<vidmFQDN>/SAAS/admin/
   
   
2. Navigate to the Catalog > Settings > Remote App Access
3. Locate the NSX Appliance name you supplied in NSX's vIDM config
4. Click on the NSX entry in the table to display the Configuration for its OAuth 2 Client. You should see Access Token Time-To-Live (TTL) set to 15 minutes.  
5. Click on EDIT next to the pencil icon and change the Access Token Time-To-Live value from 15 minutes to 16 hours.
6. Select Save to update the configuration.

 Note: This change will be OVERWRITTEN if vIDM configuration on NSX is updated in which case the steps can be performed again. Once implemented users will only need to re-enter their credentials every 16 hours (rather than 15 minutes)

Additional Information

• Even after the upgrade to the fix version, vIDM configuration needs to be refreshed as per the steps in the resolution.
• The session timeout on the NSX UI (get service http) also needs to be updated to 16 hours If using the local NSX login page to login.