UTF8 Portuguese Characters are blocked by Access Gateway with HTTP error 500.

Products

• SITEMINDER

Issue/Introduction

When accessing an application through access gateway, user requests a pdf page with Portuguese characters names, the page can not be displayed, and user gets HTTP server error 500.

Web agent trace log:

[07/01/2022][11:18:57][9593][140645540796160][][CSmHttpPlugin::ValidUrlString][URL contains BadUrlChars: '/path/filename_.....clara%C3%A7%C3%A3o+banc%C3%A1rios+......go.pdf&file=n*&hmimetype='.]
[07/01/2022][11:18:57][9593][140645540796160][2452721e-2453bec4-d662e3ff-2b7b7926-2502da07-8a][CSmResourceManager::ProcessResource][SM_WAF_HTTP_PLUGIN->ProcessResource returned SmExit.]
[07/01/2022][11:18:57][9593][140645540796160][2452721e-2453bec4-d662e3ff-2b7b7926-2502da07-8a][CSmResourceManager::ProcessResource][Calling SM_WAF_AG_PLUGIN->ProcessResource.]
[07/01/2022][11:18:57][9593][140645540796160][2452721e-2453bec4-d662e3ff-2b7b7926-2502da07-8a][CSmResourceManager::ProcessResource][SM_WAF_AG_PLUGIN->ProcessResource returned SmNoAction.]
[07/01/2022][11:18:57][9593][140645540796160][2452721e-2453bec4-d662e3ff-2b7b7926-2502da07-8a][CSmResourceManager::ProcessResource][Plugins did not collect required resource data.]
[07/01/2022][11:18:57][9593][140645540796160][2452721e-2453bec4-d662e3ff-2b7b7926-2502da07-8a][ProcessRequest][ResourceManager returned SmExit, end new request.]
[07/01/2022][11:18:57][9593][140645540796160][][ReportHealthData][Accumulating HealthMonitorCtxt.]
[07/01/2022][11:18:57][9593][140645540796160][2452721e-2453bec4-d662e3ff-2b7b7926-2502da07-8a][Tomcat5SerializedAgentData::doResponse][HTTP Status Code = 500]

Environment

Release : 12.8

Component : SITEMINDER SECURE PROXY SERVER

Cause

Portuguese characters are UTF8 Characters.
For example:
%C3%A7 translates to ç 
%C3%A3 translates to  ã

If you check ascii table, you will find them are outside range of %00-%7F.

 

Current SiteMinder ACO BadUrlChars will block any chars between %7f-%ff.

%C3 and %A7 are among the blocked list.
[9593/140647943976704][Tue Jun 28 2022 12:48:24] badurlchars=./,/*,*.,\,%00-%1f,%7f-%ff,%25

Documentation specifies that when configuring ACO BadUrlChars, there is a limitation:

Limits:
The default hexadecimal numbers apply to English characters. For other languages, remove any hexadecimal values that correspond to the characters of the language that you want to allow. Examples of such languages include (but are not limited to), Brazilian Portuguese, French, Japanese, and Chinese.

Basically the badurlchars needs to be adjusted to let through UTF8 Characters hexadecimal encoding.

Resolution

Adjust ACO BadUrlChars to allow Portuguese characters. 
The proposed solution is: badurlchars=./,/*,*.,\,%00-%1f,%25

Recycling access gateway service after the change.

Additional Information

https://www.ascii-codes.com/cp860.html

https://techdocs.broadcom.com/us/en/symantec-security-software/identity-security/siteminder/12-8/configuring/web-agent-configuration/user-protection/help-prevent-attacks.html#concept.dita_a4f1c9b394b5b45650256db5105c5886a242345b_SpecifyBadURLCharacters