ProxySG- AD account keep getting locked by BCAAA Service when using IWA-BCAAA authentication

Products

• ProxySG Software - SGOS

Issue/Introduction

Domain account keep getting locked by BCAAA Service. Based on the AD server log, the account being locked by process name bcaaaa-realm.exe.

Environment

ProxySG/ASG/ISG 

SGOS 6.7.x.x and later

BCAAA-: 6.1.4 and later

Cause

Account lockouts if showing the source as proxy/BCAAA is caused by wrong credentials being shared back by applications using the proxy. 
This could be caused any machine using wrong (or old) credentials of a user. 
If this is happening very frequently for specific users, do check their machine to see whether any application other than browser is having saved credentials

Resolution

1. Enable BCAAA debugs

Refer https://knowledge.broadcom.com/external/article/166076/gather-bcaaa-debug-logs.html

2. In the debug logs, check for the following error

2022/01/20 08:08:24.403 [6428] [6976:6428] Failed NTLM Authentication for user: 'DOMAIN\adminuser'; status=1909:0x775:The referenced account is currently locked out and may not be logged on to.

3. Check the Eventlogs on ProxySG to identify the machine ip 

Go to https://<proxy-ip>:8082/Eventlog/download/events.log 
Save the output as eventlog.txt

Search for DOMAIN\adminuser

Event log will show

Authentication failed from <xx.xx.xx.xx>: user 'DOMAIN\adminuser', realm='<realm-name>'

4. Alternative option to identify machine ip address

To find the computer that is causing this issue, a Policy Trace of the authentication requests made to the ProxySG will be necessary.

Use the following steps to set up this trace: 

. Open VPM within the Management Console.

. Open the Web Authentication Layer that holds the IWA-BCAAA  based Authentication Rules.

. For each rule in this layer that authenticates users, a trace will be added:

   * In the track cell for the authentication rule, right click and choose Set.

   * Provide a name for the trace (this is what you will see in the rule).

   * Ensure Verbose Tracing is checked.

   * Provide a filename of the trace (this is what the actual file will be called).

   * Click OK, then OK at the previous window.

. Add this same trace to each rule within the Web Authentication Layer that would be responsible for authenticating this particular user.

All authentication requests that are made by the ProxySG will now be logged to this trace file. After the user lockout occurs, you will then be able to search this trace file for all transactions related to that particular user and then determine what IP addresses the requests came from, narrowing the search for the service that has the expired/invalid credentials

*Caution*

It is highly recommended that the ProxySG be monitored closely while this Trace is active, as it will require more resources than the ProxySG typically uses. Ensure that the Trace will not be too resource intensive on the ProxySG before letting it run unattended for long periods of time.

5. Investigate on the machine xx.xx.xx.xx identified in step-3 and/or step-4