Complex (2-byte) characters in password do not work when calling AE Rest API

Products

• CA Automic One Automation

Issue/Introduction

As soon as a password for a User on AWA contains a complext character (ie. the paragraph symbol §), this will work fine to login via the AWI but not via the AE Rest API.

This results in the error "Credentials invalid".

This error occurs regardless of whether the password is passed in plain text or encrypted via base64.

Environment

Release : 12.x and 21.x

Component: Automation Engine

Context: Rest API Authentication

Cause

Encoding issue of passwords containing 2 Byte characters having as well another encoding on 1 Byte in ISO-8859-1 (what Rest API is using).

Resolution

The problem is that the paragraph sign § had a different encoding in UTF-8 (0xc2b6 - two bytes) and 0x14 one byte in ISO-8859-1. This explains why it works with AWI and it does not with Rest-API.

With Basic Authentication, the Header looks like (in plaintext)
Authentication: Basic (PWTEST/ENG:12§34) - this gets encoded in Base64

So when using ISO-8859-1 the Base64 encoding for the User/Dep:Pass string is:

UFdURVNUL0VORzoxMqczNA== 

When you use UTF-8 the encoding looks like

UFdURVNUL0VORzoxMsKnMzQ=

Note the difference in the last chars.

Postman and all browsers default to UTF-8, and that is why the login attempt per default does not work.

According to the related RFC 2617 and/or RFC 7617 the default encoding is still undefined, it needs to be compatible with ASCII.

The REST interface on the other hand evaluates the headers as ISO, and that is why the password does not work.

To make this character work in the password field, the workaround is to use create the Http header authorization manually, calculate the base64 encoding of the 'USER/DEPARTMENT:PASSWORD' as ISO-8859-1 chars and add this as a value for the Authorization header in Postman.

This can be done easily with notepad++, see here how:

and the result will be replaced in the same tab.

Else, this can also be performed on Linux, example:

echo -n username:password | iconv -t ISO-8859-1 | base64

 

Additional Information

The corresponding part in RFC 7617 about this:

 

The original definition of this authentication scheme failed to specify the character encoding scheme used to convert the user-pass into an octet sequence. In practice, most implementations chose either a locale-specific encoding such as ISO-8859-1 ([ISO-8859-1]), or UTF-8 ([RFC3629]). For backwards compatibility reasons, this specification continues to leave the default encoding undefined, as long as it is compatible with US-ASCII (mapping any US-ASCII character to a single octet matching the US-ASCII character code).