How to handle NTLM Authentication during recording and playback in VSE

Products

• Service Virtualization

Issue/Introduction

The record and playback of the NTLM authenticated server are not straightforward in DevTest.

NTLM is actually a 3 request handshake where the client (browsers, apps, etc.) don't send any credentials on their first request for a resource. This means that the first request is anonymous, even if credentials have been configured for that resource. This anonymous request, when Windows Auth is enabled and Anonymous Auth is disabled, results in an HTTP 401 status.

The 2nd request would be the NTLM challenge where the client re-sends the original request with an additional "Authorization" header, containing the NTLM Type-1 message. The server then sends the NTLM challenge (Type-2 message) back to the client with HTTP 401 status.

The 3rd request would be the original request where the client sends it one more time by adding the challenge response (NTLM Type-3 message) in the "Authorization" header. The server then authenticates the user and sends back the response with HTTP 200 status (if successful). 

See below:

1. Anonymous Request

Request#1: 

GET /beneficiaries HTTP/1.1

Host: server

username: user1

password: password1

 

Response#1

WWW-Authenticate: Negotiate

WWW-Authenticate: NTLM

 

2. NTLM Challenge

Request#2:

GET /beneficiaries HTTP/1.1

Host: server

username: user1

password: password1

Authorization: Negotiate TlRMTXXXXXXXXXXXXAAAAAAAAAAAAAAGA4AlAAAADw==

 

Response#2

HTTP/1.1 401 Unauthorized

Content-Length: 341

WWW-Authenticate: Negotiate TlRMTVNTUAACAAAADAAMADgAAAAVgoXXXXXXXXXXXXXXXXXXXXXXXXXXAAAA9DAEcAVQBTAEUAUgACAAwAQwBHAFUAUwBFAFIAAQAOAFcANgAyADgAOAAwADgABAAmAGMAZwB1AHMAZQByAC4AYwBhAHAAZwByAG8AdQBwAC4AYwBvAG0AAwA2AHcANgAyADgAOAAwADgALgBjAGcAdQBzAGUAcgAuAGMAYQBwAGcAcgBvAHUAcAAuAGMAbwBtAAUAGABjAGEAcABnAHIAbwB1AHAALgBjAG8AbQAHAAgAam094xJI1wEAAAAA

 

3. Actual Request

Request#3:

GET /beneficiaries HTTP/1.1

Host: server

username: user1

password: password1

Authorization: Negotiate TlRMTVNTUAADAAAAGAAYAJgAAABaAVoBsAAAAAwADABYAAAAJgAmAGQAAAAOAA4AigAAABAAEAAKAgAAFYKI4gYDgCUAAAAPYtqAVIATgz5y2e67FLd8pUMARwBVAFMARQBSAE8AWgBUAFMAVABBAEEAMAAwADAAMAAxADcAMgAxAFMAMAAwADEAVwAzADQAMAA2ADUANwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACghX5LlC0oXHRV/uPp2MnpAQEAAAAAAABqbT3jEkjXATnmh7qPHrK3AAAAAAIADABDAEcAVQBTAEUAUgABAA4AVwA2ADIAOAA4ADAAOAAEACYAYwBnAHUAcwBlAHIALgBjAGEAcABnAHIAbwB1AHAALgBjAG8AbQADADYAdwA2ADIAOAA4ADAAOAAuAGMAZwB1AHMAZQByAC4AYwBhAHAAZwByAG8AdQBwAC4AYwBvAG0ABQAYAGMAYQBwAGcAcgBvAHUAcAAuAGMAbwBtAAcACABqbT3jEkjXAQYABAACAAAACAAwADAAAAAAAAAAAAAAAAAwAAAG+OxnfGdDmcmSYktcAPW+7iXAgsaB2zsVd1OQNLENmQoAEAAVctazkdh/oJb8rq1FjeP5CQAkAEgAVABUAFAALwAxADAALgAyADIANgAuADEAOQAuADEANAAwAAAAAAAAAAAA6xNc+udjP6jjRINUdwT5xg==

 

Response#3

HTTP/1.1 200 OK

{
"accountNumber": "123456789",
"name": "James",
"id": "b3898288-bb6b-4a79-a469-be45e9a06f80"
}

Environment

Release : 10.6

Component : CA Service Virtualization

Cause

As NTLM authenticated web service makes 3 requests (in fact all are the same 'EXACT' requests including arguments in the 'DevTest' world) for a single transaction with(or without) Authorization header depending on the response during the handshake, the current default matching logic in DevTest would fail to give the correct responses during playback. Also, it depends on how the client is executing the calls during playback. It can be a full handshake(3 requests) for every call or 2 requests by preemptively sending the NTLM web authentication type-1 message or just one request by sending NTLM type-2 message directly( as obtained from a separate login call) without repeating handshake. So it becomes challenging for DevTest to perform the matching of incoming requests from clients with an appropriate response.

 

Resolution

Add an explicit matching logic to handle NTLM authentication during playback using any of the below methods. 

1. Match script in VSI

Record the NTLM authenticated web service as normal and add the below separate matching script logic for each type of request in the transaction in the "Match Script" window in VSI after recording as shown below:

Request#1 (Anonymous Request)

 

Request#2 NTLM Challenge

 

Request#3 Original Request

 

 

2. Scriptable DPH

During recording, add a secondary DPH to JSON on the request data protocol called "scriptable DPH" with the below code:

------

%beanshell%

import com.itko.util.ParameterList;

import com.itko.util.Parameter;

import com.itko.lisa.vse.stateful.model.ArgumentType; 

 

incomingMetaData = lisa_vse_request.getMetaData();

ParameterList args = lisa_vse_request.getArguments();

 

if (!incomingMetaData.containsKey("Authorization") ||  !incomingMetaData.get("Authorization").startsWith("Negotiate")) {

args.addParameter(new Parameter("auth", null));

}

else if (incomingMetaData.containsKey("Authorization") && incomingMetaData.get("Authorization").startsWith("Negotiate"))

{

authValue = incomingMetaData.get("Authorization");

result = new sun.misc.BASE64Decoder().decodeBuffer(incomingMetaData.get("Authorization").substring(10)); 

if (result[8]==1) {

args.addParameter(new Parameter("auth", authValue));

} else if (result[8]==3) {

ArgumentType type = new ArgumentType();

type.setOperator(ArgumentType.Operator.ANY);

args.addParameter(new Parameter("auth", authValue, type));

//args.addParameter(new Parameter("auth", authValue));

}

}

lisa_vse_request.setArguments(args);

------------

Note: 

Make sure "Allow Duplicate Specific Transaction" is set to false during recording.

Any change in DPH requires re-recording.

 

How to Test/Playback

Perform full handshake (3 requests) as shown below and verify the responses:

Request#1 (Anonymous Request)

GET http://{{Server}}:12345/api/v2/investor-accounts/58087638/beneficiaries

Expected Response:

HTTP/1.1 401 Unauthorized

NO DATA

 

Request#2 (NTLM Challenge)

GET http://{{Server}}:12345/api/v2/investor-accounts/58087638/beneficiaries

Authorization: Negotiate TlRMTVNTUAABAAAAl4II4gAAAAAAAAAAAAAAAAAAAAAGA4AlAAAADw==

Expected Response:

HTTP/1.1 401 Unauthorized

<HTML><HEAD><TITLE>Not Authorized</TITLE>
<META HTTP-EQUIV="Content-Type" Content="text/html; charset=us-ascii"></HEAD>
<BODY><h2>Not Authorized</h2>
<hr><p>HTTP Error 401. The requested resource requires user authentication.</p>
</BODY></HTML>

 

Request#3 (Original Request)

GET http://{{Server}}::12345/api/v2/investor-accounts/58087638/beneficiaries

Authorization: Negotiate TlRMTVNTUAADAAAAGAAYAJgAAABaAVoBsAAAAAwADABYAAAAJgAmAGQAAAAOAA4AigAAABAAEAAKAgAAFYKI4gYDgCUAAAAPYtqAVIATgz5y2e67FLd8pUMARwBVAFMARQBSAE8AWgBUAFMAVABBAEEAMAAwADAAMAAxADcAMgAxAFMAMAAwADEAVwAzADQAMAA2ADUANwAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAACghX5LlC0oXHRV/uPp2MnpAQEAAAAAAABqbT3jEkjXATnmh7qPHrK3AAAAAAIADABDAEcAVQBTAEUAUgABAA4AVwA2ADIAOAA4ADAAOAAEACYAYwBnAHUAcwBlAHIALgBjAGEAcABnAHIAbwB1AHAALgBjAG8AbQADADYAdwA2ADIAOAA4ADAAOAAuAGMAZwB1AHMAZQByAC4AYwBhAHAAZwByAG8AdQBwAC4AYwBvAG0ABQAYAGMAYQBwAGcAcgBvAHUAcAAuAGMAbwBtAAcACABqbT3jEkjXAQYABAACAAAACAAwADAAAAAAAAAAAAAAAAAwAAAG+OxnfGdDmcmSYktcAPW+7iXAgsaB2zsVd1OQNLENmQoAEAAVctazkdh/oJb8rq1FjeP5CQAkAEgAVABUAFAALwAxADAALgAyADIANgAuADEAOQAuADEANAAwAAAAAAAAAAAA6xNc+udjP6jjRINUdwT5xg==

Expected Response:

HTTP/1.1 200 OK

{ "context": { "messages": [], "identity": { "conversationId"
.....
}

Artifacts

. Raw Traffic file: NTLM_raw_traffic.xml - use this sample file to record the NTLM authenticated service.

. Sample NTLM Project: NTLM_VS.zip - contains CustomMatching.vsi and CustomMatching.vsm for solution#1, NTLM_VS.vsi and NTLM_VS.vsm for Solution#2 along with AuthMatcher.tst for testing in playback  mode.

. Sample Postman Collection: NTLM_DevTest.postman_collection.json - contains postman collection for testing both solutions in playback mode. (Preferred over AuthMatcher.tst)