ODATA pull from DWH_INV_INVESTMENT failing on unicode vertical tab character

Products

• Clarity PPM SaaS

Issue/Introduction

ODATA pull from DWH_INV_INVESTMENT is failing with invalid whitespace character:

U+000B

Environment

Release : 16.0.3

Component : Clarity Data Warehouse

Resolution

Clarity user copied and pasted from another application with text formatting into a Clarity text field.

One of those characters was a VT vertical line tab character, which breaks the XML formatting returned from the Odata Service

Vertical Line Tab Unicode Character

Narrow down the pull from the Odata request until the problem is reproducible.  Add columns one by one into the $select argument of the URL.

Once the column is identified run a query like the following to find the invalid white space character.

The query below will work for the VT character only

select * from inv_investments where name ~ '%U+000B%';

 

 

The 0x0B (VT) character is not compatible with XML. See Extensible Markup Language (XML) 1.0 (Fifth Edition) (https://www.w3.org/TR/xml/#charsets) .
OData 2 using XML cannot be used to return data that contains that character. It is a limitation of XML, not of our implementation.

OData 2 can support json and json can be used to return a VT character. 
The user just needs to indicate in the query request that they want the response as json. This can be done either as query parameter or as header.

For example:  Accept: application/json

-- OR -- 

as part of the URL, for example:

https://example.com/api/odata/datasource?$format=json