Configuring Kerberos (Negotiate) Authentication in ITMS and Understanding Firewall Identity Behavior

Products

• IT Management Suite

Issue/Introduction

Organizations using IT Management Suite (ITMS) may observe authentication inconsistencies when using Kerberos (Negotiate) authentication in IIS. In some environments, network security devices such as Palo Alto firewalls may attribute traffic to a service or system account instead of the logged-in user when they cannot correctly interpret a Kerberos-authenticated exchange. This can lead administrators to switch IIS authentication to NTLM only to restore identity visibility, even though that does not address the underlying Kerberos visibility behavior.

Symptoms

• Firewall or network monitoring tools report a service account, system identity, or other non-user identity instead of the logged-in user.
• User-based applications that rely on user identity may not function as expected.
• Symantec Management Agents (SMA) may fail to register with Task Servers in some environments.
• Errors may appear such as:
  • Authentication failed, request level authentication is not supported
  • HTTP 0x80042D21

Environment

IT Management Suite 8.6 RU3, 8.7.x, and later

IIS with Windows Authentication enabled

Domain-joined environments where Kerberos is expected

Environments using traffic inspection or identity-aware firewalls

Cause

ITMS uses Windows Integrated Authentication for agent and browser communication with IIS-hosted components. On IIS, the Windows Authentication module supports both Kerberos and NTLM through the providers list, with Negotiate typically attempting Kerberos first and falling back to NTLM when Kerberos cannot complete. Microsoft documents this provider model and the authPersistNonNTLM behavior in IIS.

A failed Kerberos attempt followed by a successful NTLM authentication does not by itself indicate an ITMS defect or issue. In many cases, the actual issue is that a firewall or intermediary device cannot properly interpret Kerberos tickets and therefore reports traffic under a service, machine, or other non-user identity. 

The usual main reasons for concerns regarding Kerberos authentication has been:

1. Windows Authentication Negotiation

The Symantec Management Agent (SMA or Altiris Agent) or browser client initiates the HTTP(S) request, but protocol selection between Kerberos and NTLM is handled by Windows SSPI and IIS Windows Authentication, not by custom ITMS logic. IIS supports both Kerberos and NTLM through the Windows Authentication providers configuration.

2. Kerberos Identity Visibility

Kerberos uses service tickets tied to SPNs. Because the ticket exchange is not the same as an NTLM challenge/response flow, intermediary devices may not be able to identify the end user from the traffic and may instead attribute the traffic to a service, machine, or unknown identity. This is a visibility/interpretation issue, not necessarily an ITMS authentication problem.

3. IIS Provider Order and Fallback

When Negotiate is first in the provider list, Kerberos is attempted first. If Kerberos fails, NTLM can be used as fallback if NTLM remains enabled.
KB January Cumulative Security updates prevent endpoints from registering to task server also notes that putting NTLM on top of Negotiate may be required in some environments, especially with non-domain clients or CEM scenarios.

4. SPN and Application Pool Identity Requirements

Kerberos requires valid SPNs that match the identity IIS uses to decrypt the Kerberos ticket. If IIS is using the machine account, the SPNs belong on the computer object. If IIS is using a custom application pool identity and useAppPoolCredentials=True, the SPNs must be registered to that account. Microsoft explicitly documents useAppPoolCredentials for these scenarios. ITMS does not automatically create all required HTTP SPNs in Active Directory as a general product behavior; they must be validated in AD and configured to match the IIS identity model in use.

5. authPersistNonNTLM

Microsoft documents authPersistNonNTLM as controlling whether IIS reauthenticates every non-NTLM request on the same connection. Setting it to true allows IIS to cache the Kerberos token for the TCP session. In ITMS Task Server scenarios for example, this setting can be important and may contribute to failures when left at false, but it should not be described as the only requirement for Kerberos to function.

Resolution

Note: The information contained in this article is provided as a reference. Configuring Windows Authentication (Negotiate/Kerberos) is a complex infrastructure process that should be reviewed under Microsoft guidance and best practices. ITMS utilizes standard industry recommendations for these protocols.

 

Table of Contents

• When Is This Behavior Expected vs Problematic
• Authentication Flow Reference
• Recommendations / Areas to Verify
  • 1. Verify IIS Windows Authentication Is Enabled
  • 2. Verify Provider Order
  • 3. Verify Application Pool Identity
  • 4. Validate SPNs
  • 5. Check for Duplicate SPNs
  • 6. Configure useAppPoolCredentials Only When Needed
  • 7. Configure authPersistNonNTLM
  • 8. Validate Kerberos on the Client
  • 9. Validate Network and Firewall Handling
• Microsoft-Recommended Kerberos/Negotiate Implementation Considerations
• Task Server Registration Validation
• Log Locations and Interpretation
  • Server-Side Logs
  • Agent-Side Logs
  • Log Interpretation Guidance
• Workaround
• Common Misconceptions
• Key Takeaways

When Is This Behavior Expected vs Problematic

Scenario	Expected	Action Required
Firewall shows a service/system account, but ITMS functions correctly	Yes	Usually no
Kerberos attempt fails, then NTLM succeeds	Yes	Optional tuning
Agents cannot register with Task Server	No	Investigate
Repeated authentication errors appear in logs	No	Investigate
Kerberos is required because NTLM is disabled	No	Full Kerberos validation required

---

Authentication Flow Reference

Negotiate → Kerberos attempt → (if Kerberos cannot complete) → NTLM fallback

A successful NTLM fallback after a failed Kerberos attempt is normal Windows authentication behavior when both providers are enabled.

 

Recommendations / Areas to Verify

1. Verify IIS Windows Authentication Is Enabled

Microsoft recommends enabling Windows Authentication on the relevant IIS site, application, or virtual directory and disabling anonymous access where Windows Authentication is required.

Steps:

1. Open IIS Manager.
2. Browse to the relevant ITMS site or virtual directory.
3. Open Authentication.
4. Confirm Windows Authentication is Enabled.
5. Confirm Anonymous Authentication is configured appropriately for the scenario.

2. Verify Provider Order

1. In Authentication, select Windows Authentication.
2. Click Providers.
3. Ensure the providers are ordered as follows for Kerberos-capable domain clients:

Provider	Recommended Order
Negotiate	First
NTLM	Second

Note: In some non-domain, internet-facing, or CEM-related scenarios, NTLM may need to remain enabled or even be prioritized as a workaround. Broadcom notes this possibility in KB January Cumulative Security updates prevent endpoints from registering to task server.

3. Verify Application Pool Identity

1. Open IIS Manager.
2. Go to Application Pools.
3. Identify the pool used by the relevant ITMS application.
4. Open Advanced Settings.
5. Review the Identity setting.

Identity Type	Kerberos Consideration
Machine account / default behavior	SPNs on computer object
Custom domain account	SPNs on that account, with useAppPoolCredentials=True

Microsoft’s IIS Kerberos guidance for custom identities aligns with this requirement.

4. Validate SPNs

Run the following on a domain-joined system with appropriate rights:

setspn -L <ServerNameOrServiceAccount>

Validate the HTTP SPNs expected for the SMP or Task Server host name, such as:

 

HTTP/servername
HTTP/servername.example.com

KB January Cumulative Security updates prevent endpoints from registering to task server also references setting SPNs as part of the mitigation for Kerberos-related registration issues based on recent changes done by Microsoft and Kerberos authentication.

1. Identify the Identity: 

   • Go to Application Pools, right-click DefaultAppPool (or the one running Altiris),

   • and select Advanced Settings.
     Note the Identity (e.g., DOMAIN\AltirisService).

2. Set SPNs: Open an elevated Command Prompt and register the HTTP SPNs for the FQDN and NetBIOS name of the SMP Server:

   setspn -s HTTP/SMP-ServerName DOMAIN\AltirisService
   setspn -s HTTP/SMP-ServerName.domain.com DOMAIN\AltirisService

5. Check for Duplicate SPNs

Duplicate SPNs can break Kerberos resolution.

Run the following from a command prompt as administrator:

setspn -X

If duplicates are found, correct them before retesting Kerberos. This is a standard Kerberos validation step consistent with Microsoft troubleshooting practice.

6. Configure useAppPoolCredentials Only When Needed

If IIS uses a custom domain account for the application pool:

1. Open Configuration Editor in IIS.
2. Navigate to:
   system.webServer/security/authentication/windowsAuthentication
3. Set:
   useAppPoolCredentials = True

Microsoft specifically recommends this when the SPN is tied to the app pool identity rather than the computer account.

7. Configure authPersistNonNTLM

1. Open Configuration Editor in IIS:
   • Open IIS Manager on the server being connected to (Notification Server or Site Server).
   • Select the ServerName under Start Page, and then select Configuration Editor (near the bottom)
2. Navigate to:
   system.webServer/security/authentication/windowsAuthentication
3. Set:
   authPersistNonNTLM = True

Microsoft documents that true causes IIS to authenticate the non-NTLM client once per TCP session instead of reauthenticating every request. This can help with Kerberos-based ITMS communication scenarios, including Task Server registration issues described in KB Error from logs: Authentication failed, request level authentication is not supported, check that IIS setting 'authPersistNonNTLM' is set to 'True' on the server.

8. Validate Kerberos on the Client

On the client, after reproducing the connection:

Run the following from a command prompt as administrator:

klist

Look for an HTTP service ticket for the target server. This is a Microsoft-aligned validation step for confirming Kerberos use.

9. Validate Network and Firewall Handling

Check:

1. DNS resolution of the SMP Server or Task Server hostname.
2. HTTPS access to the expected URL.
3. SSL/TLS inspection policy on intermediary devices.
4. User-ID or AppID handling on the firewall.

If a firewall cannot interpret Kerberos traffic correctly, it may be necessary to:

• Exempt the ITMS URL from SSL decryption where appropriate
• Avoid AppID overrides that interfere with authentication
• Adjust user identity mapping expectations

Firewalls can block traffic even if the required ports are open.

• Verification: Open a browser and navigate to https://<SMPSERVER_FQDN>:443. If the IIS welcome page does not display, the firewall is likely filtering URL traffic.
• Action: Create a rule on your firewall (such as the Palo Alto) device to explicitly allow URLs for the SMP Server to pass through Error adding SMP Server.

This part should be described as a network-device interpretation issue, not as proof that ITMS authenticated with the wrong account.

---

Microsoft-Recommended Kerberos/Negotiate Implementation Considerations

The following Microsoft-aligned points should be included because they help frame the IIS side correctly:

1. Install and enable Windows Authentication in IIS where required.
2. Use Negotiate when Kerberos is desired, while leaving NTLM available if fallback is required.
3. Ensure the SPN matches the account IIS uses for decryption.
4. Use useAppPoolCredentials=True only when the app pool runs under a custom account and SPNs are registered to that account.
5. Consider authPersistNonNTLM=True when repeated Kerberos reauthentication affects the application flow.
6. Validate Kerberos with klist, SPNs with setspn, and authentication outcomes through IIS/application logs and network traces.

---

Task Server Registration Validation

If the issue involves Task Server registration:

1. Review the IIS Windows Authentication settings on the Task Server virtual directory or site used for agent communication.
2. Confirm provider order and authPersistNonNTLM.
3. Validate the SPNs for the Task Server host name.
4. Confirm the client can obtain the expected Kerberos ticket.
5. If Kerberos fails but NTLM succeeds, determine whether NTLM fallback is acceptable for the environment or whether Kerberos must be enforced.

KB January Cumulative Security updates prevent endpoints from registering to task server specifically ties January 2022 changes and SPN/IIS adjustments to endpoint registration issues.

---

Log Locations and Interpretation

Server-Side Logs

Common server-side locations include:

• C:\ProgramData\Symantec\SMP\Logs
• C:\inetpub\logs\LogFiles

Agent-Side Logs

For agent-side troubleshooting, use the Symantec Management Agent log folder:

• C:\ProgramData\Symantec\Symantec Agent\Logs\axxx.log

Log Interpretation Guidance

Log / Evidence	Interpretation
Authentication failed	Authentication negotiation failed
request level authentication is not supported	IIS/authentication persistence/config mismatch may exist
IIS 401 responses	Authentication did not complete successfully
Kerberos fails, NTLM succeeds	Expected fallback when both providers are enabled
Firewall shows service/system identity	Often device interpretation, not proof of wrong ITMS credentials

---

Workaround

If the environment cannot support Kerberos correctly or the network security tooling requires NTLM for identity visibility, IIS can be configured to prioritize or use NTLM.

Advantage	Disadvantage
Improves identity visibility in some tools	Reduces security posture
Simplifies some non-domain scenarios	Disables or bypasses Kerberos benefits

This should be positioned as a workaround or compatibility choice, not as the preferred long-term configuration when Kerberos is required. 

---

Common Misconceptions

Misconception	Correct Interpretation
ITMS selects Kerberos or NTLM	Windows SSPI and IIS do
Seeing a service account means ITMS authenticated incorrectly	Often false; it may be a visibility issue
ITMS automatically creates the needed SPNs	SPNs must be validated and managed in AD
NTLM is always required for ITMS	No; it depends on the environment
Kerberos failure followed by NTLM success always means a defect	No; it is expected fallback behavior

---

Key Takeaways

• Kerberos vs NTLM selection is controlled by Windows SSPI and IIS, not by custom ITMS authentication logic.
• A firewall showing a service or system identity does not by itself prove incorrect ITMS authentication.
• authPersistNonNTLM=True can be important for some ITMS Kerberos scenarios, but it is not the sole Kerberos requirement.
• SPNs must match the IIS identity model in use.
• Switching to NTLM can be a practical workaround, but it should be documented as a tradeoff, not as the root-cause fix.

Additional Information

Error from logs: Authentication failed, request level authentication is not supported, check that IIS setting 'authPersistNonNTLM' is set to 'True' on the server

ITMS 8.x - After Server restart you notice a Warning event in SMP Logs regarding SPN configuration not being registered Error Code 8344

Setup Symantec Management Platform environment to use Kerberos authentication if NTLM is disabled

January Cumulative Security updates prevent endpoints from registering to task server