Error: "The referenced account is currently locked out " - Edge SWG (ProxySG) - AD domain account locked out when using IWA-BCAAA authentication

Products

• ProxySG Software - SGOS
• ISG Proxy
• Advanced Secure Gateway Software - ASG

Issue/Introduction

Domain accounts keep getting locked out by the BCAAA Service. The  Active Directory (AD) server log indicates the domain account is being locked out due to the process name bcaaaa-realm.exe. 

Environment

BCAAA version 6.1.4 and later

Cause

If the BCAAA service is showing as the cause of the lockout in the AD server logs, applications are sending the wrong credentials to the Edge SWG.
This could be caused any machine using the wrong (or old) user credentials. 
If this is occurring frequently for specific users, check their machine to see if any applications (besides the browser) are using saved credentials.

Resolution

To resolve this issue, you must identify the offending users and machines then remedy the issue (incorrect/old credential) on the end user machine. Here is a list of steps to find the users or machines with this issue:

1. 1. Enable BCAAA debugs as mentioned in this KB - gather BCAAA debug logs 
   2. In the debug logs, look for the errors similar to:
      • 2022/01/20 08:08:24.403 [6428] [6976:6428] Failed NTLM Authentication for user: 'DOMAIN\adminuser'; status=1909:0x775:The referenced account is currently locked out and may not be logged on to.
   3. Check the Eventlogs on Edge SWG to identify the machine IP:
      1. Go to https://<proxy-ip>:8082/Eventlog/download/events.log and save the output as eventlog.txt
      2. Search for the locked out user, in our example its: DOMAIN\adminuser
      3. Event log will have a message like:
         • Authentication failed from <xx.xx.xx.xx>: user 'DOMAIN\adminuser', realm='<realm-name>'
   4. Alternatively, identify the offending machine's IP address using a Policy trace to trace authentication requests. To set up the Policy trace:
      1. Open the Web VPM within the Management Console.
      2. Open the Web Authentication Layer that holds the IWA-BCAAA  based Authentication Rules.
      3. For each rule in this layer that authenticates users, a trace will be added:
         1. In the "Track" cell for the authentication rule, right-click and choose Set.
         2. Provide a name for the trace (this is what you will see in the rule).
         3. Ensure Verbose Tracing is checked.
         4. Provide a filename of the trace (this is what the actual file will be called).
         5. Click OK, then OK on the previous window.
      4. Add this same trace to each rule within the Web Authentication Layer responsible for authenticating this particular user.

         • All authentication requests by the Edge SWG will now be logged to this trace file. After the user lockout occurs, you will then be able to search this trace file for all transactions related to that particular user and then determine what IP addresses the requests came from, narrowing the search for the service that has the expired/invalid credentials

           Caution: Monitor the Edge SWG closely while this Trace is active, as it will require more resources than it typically uses. Verify the Policy trace will not be too resource intensive on the Edge SWG before letting it run unattended for long periods of time.

   5. Investigate the machines identified in steps 3 or 4. Check the machine to see if any applications use saved or incorrect credentials.