Troubleshooting Schannel bottlenecks and "Resetting Schannel" errors with IWA-Direct

Products

ProxySG Software - SGOS

Issue/Introduction

When using IWA-Direct authentication on the Edge SWG (formerly ProxySG), users may experience authentication delays or failures. In the event logs, you may observe "Schannel Reset" errors indicating a bottleneck in communication with the Domain Controller (DC).

Symptoms include:

Event Log Errors: "Schannel (<Domain>): Resetting Schannel due to error: 0xC000020D(-1073741299)"
High Latency: Users experience slow page loads. Policy traces show high latency in the client-in checkpoint or authentication stage (e.g., authentication start 780 elapsed 19155 ms or client-in: start 780 elapsed 19155 ms).
LSA Stats: The Local Security Authority (LSA) statistics show a high number of "Current Waiters."

Environment

Edge SWG (ProxySG)
IWA-Direct Authentication Realm

Cause

High latency with authentication can potentially indicate that the proxy has exhausted its available secure channel (Schannel) slots used to communicate with the Domain Controller.s

This bottleneck often occurs when one or more of these conditions are true:

High NTLM Load: NTLM authentication is being used heavily instead of Kerberos. NTLM requires a dedicated Schannel slot for the duration of the authentication, whereas Kerberos does not.
Concurrency Limit Reached: The configured max-secure-channel-requests on the Edge SWG configured low compared to the volume of incoming authentication requests.
Configuration Mismatch: The MaxConcurrentApi setting on the Domain Controller does not match the capacity configured on the Edge SWG.

Resolution

To resolve schannel bottlenecks, you must diagnose the current load and potentially increase the concurrent schannel limit and consider how to optimize the authentication flow.

1. Diagnose the Bottleneck

You can verify if the schannel is the bottleneck by viewing the LSA statistics.

Access the advanced URL: https://<Proxy_IP>:8082/lsa/stats and review the following fields:
- Current Waiters: The number of user requests currently paused in a queue waiting for a free schannel slot. During a bottleneck, this number will be high.
- Max Waiters: The highest number of queued users seen at any one time.
LSA Debug
- To obtain further debug information capture an LSA debug using the article, How to do LSA Debug on ProxySG or Advanced Secure Gateway
- Review the LSA debug for issues with the schannel connection. Typically this debug log is reviewed by Broadcom support.

2. Increase Concurrent Schannel Slots

If "Current Waiters" is high and it may be necessary to increase the number of available schannel slots.

Note: You must perform this change on both the Edge SWG and the Domain Controller(s) for it to be effective.

On the Edge SWG: Run the following commands in the CLI (adjusting the domain name and value as needed)

> enable
# conf t
# (config) security windows-domains
# (config windows-domains) edit <Domain_Alias>
# (config windows-domains <Domain_Alias>) max-secure-channel-requests 50

On the Domain Controller: You must edit the Windows Registry to increase the MaxConcurrentApi value to match the setting on the Edge SWG.

Refer to Microsoft documentation for setting MaxConcurrentApi on the Domain Controllers serving the proxy.

3. Optimize Authentication (Reduce NTLM)

Schannel bottlenecks are frequently caused by excessive NTLM traffic. Kerberos is more efficient and avoids this bottleneck.

Verify Kerberos Configuration: Ensure your IWA-Direct realm is properly configured for Kerberos.
Load Balancers: If a Load Balancer is in front of the Edge SWG, ensure it is configured to support Kerberos. If the Load Balancer does not persist the connection correctly, clients may fall back to NTLM.
Authentication Mode: Check your Authentication Mode settings.
- If set to Proxy mode, the Edge SWG performs an explicit proxy challenge on every connection.
- If possible, use Proxy-IP or Origin-IP modes, as these can be more efficient depending on the network topology.

Additional Information

After applying the changes, monitor the environment to ensure the bottleneck clears.

Advanced URL: https://<Proxy_IP>:8082/lsa/stats
CLI Monitoring: Use show advanced-url /lsa/stats
- Success criteria: Current Waiters should stay at or near 0. Schannel Resets should stop incrementing rapidly.
SNMP Monitoring: You can use the BLUECOAT-SG-AUTHENTICATION-MIB to monitor the currentWaiters OID.
Access Log Monitoring You can use Access Logs and your SIEM integration to identify authentication latency. To do this, ensure your log format includes time-tracking fields. For a list of specific log fields recommended for monitoring authentication performance, see Optimize IWA Direct Authentication (Refer to the "Report Statistics to Watch" section).

References