Generate SHA 26 Hash value not matching expected value

Products

• STARTER PACK-7
• CA Rapid App Security
• CA API Gateway

Issue/Introduction

For the input value : 

--------------------------
test

test
--------------------------

(sans indent) 

The online tool at : 

https://www.dcode.fr/sha256-hash

Generates : 
    hex: 5222d2759616f0cb5795ff2b345204a9bf40d9b6fa01f6670290a4958f81a5a9

Whereas the "Generate Security Hash" 

Generates : 
   HEX: 72EF1BCD7A4D61B0DBF76327DBFFA77E245BB020210A90C06F0BA388BF235E3A

 

Cause

As well as : 

https://www.dcode.fr/sha256-hash

which gives a different answer. 

We find another online tool

https://hash.online-convert.com/sha256-generator

That gives the same answer as the "Generate Security Hash" assertion.

After printing out the raw characters being hashed we find that the assertion is not ignoring the OA or OD characters.

After a bit of experimenting we see that if the EOL character is OA we get the match of the hash for the first tool, and if the EOL is CR LF we get a match on the hash for the second tool.

So the difference between the hashes is due to different EOL characters in the raw input data. 

 

Environment

API Gateway 9.2 
API Gateway 9.3

Resolution

We found that the difference was in the set-context variables where there was option to interpret end-of-line characters as : 

a) CR/ LF
b) LF

We find the online tool was generating hash using LF, whereas the default for "set-context variable" was CR/LF

Once we change the set-context variables to use LF as the eol character we matched the hash256 value from the website. 

Printing out the hex of the input values was useful for debugging to determine exactly what value (lf and cr and all ) is being hashed.
 

<Please see attached file for image>

 

Additional Information

Printing the hex value of the raw data that is being hashed is a very helpful way to determine which non visible characters are being used - notably here the CR and CR LF characters.  which are OD and OD OA in hex. 
 

<Please see attached file for image>

Results using CR/LF :

Hash:  72EFBFBD1BEFBFBD7A4D61EFBFBDEFBFBDEFBFBD6327EFBFBDEFBFBDEFBFBD7E245BEFBFBD20210AEFBFBDEFBFBD6F0BEFBFBDEFBFBDEFBFBD235E3A

Input: 
746573740D0A0D0A74657374

Results using LF :

Hash:  5222EFBFBD75EFBFBD16EFBFBDEFBFBD57EFBFBDEFBFBD2B345204EFBFBDEFBFBD40D9B6EFBFBD01EFBFBD6702EFBFBDEFBFBDEFBFBDEFBFBDEFBFBDEFBFBDEFBFBD

Input: 
746573740A0A74657374