NAS 9.34 or higher crashes, restarts and throws Max. restarts error - unstable nas probe

Products

• DX Unified Infrastructure Management (Nimsoft / UIM)
• CA Unified Infrastructure Management On-Premise (Nimsoft / UIM)
• CA Unified Infrastructure Management SaaS (Nimsoft / UIM)
• Unified Infrastructure Management for Mainframe

Issue/Introduction

nas stopped running again and is in an error state. nas restarts and throws Max. restarts error in log.

Jun  8 10:42:49:291 [21564] 0 nas: ****************[ Starting ]****************
Jun  8 10:42:49:291 [21564] 0 nas: nas 9.34, Nov 28 2021
Jun  8 10:42:49:291 [21564] 0 nas: Copyright  2013, CA. All rights reserved.
Jun  8 10:42:50:293 [21564] 1 nas: nimNamedSession: failed to connect session to 10.x.x.xxx:48033 10061
Jun  8 10:42:50:299 [21564] 1 nas: port=48033 PID=29752
Jun  8 10:42:50:311 [21564] 0 nas: No longer checking for restricted hub license
Jun  8 10:42:50:311 [21564] 0 nas: NAS Services called using mode: 1
Jun  8 10:42:50:312 [21564] 0 nas: Scripting license is available.
Jun  8 10:42:50:313 [21564] 1 nas: passive maint thread - has started in thread 2204
Jun  8 10:42:50:316 [21564] 0 nas: Failed to read a valid probe_crypto_mode from controller. Assuming pre-FIPS and using TWO_FISH
Jun  8 10:42:50:317 [2204] 0 nas: maint:  Successful registration to: /<domain>/<hub>/<robot>/maintenance_mode
Jun  8 10:42:50:445 [21564] 0 nas: corrInitialize: Last Alive Time: 1654677748
Jun  8 10:42:50:446 [29780] 1 nas: Transaction-log database housekeeping scheduled to Thu Jun 09 00:30, 2022
Jun  8 10:42:50:901 [18120] 1 nas: Activity-log administration used 407ms, status: OK
Jun  8 10:42:51:034 [9552] Controller: Max. restarts reached for probe 'nas' (command = nas.exe)

nas was still crashing even after update to nas version 9.36.

Cause of the crash:

There was a crash in the third-party PCRE lib when it is trying to match the regex "/^((?!PIM99|PST001).)*\$/" with the below alarm message as per the Windows Events System log:

Alarm message:

WIN06: Microsoft-Windows-DistributedCOM Error(10016 - <Event xmlns='http://schemas.microsoft.com/win/2004/08/events/event'><System><Provider Name='Microsoft-Windows-DistributedCOM' Guid='{1B562E86-B7AA-4131-BADC-B6F3A001407E}' EventSourceName='DCOM'/><EventID Qualifiers='0'>10016</EventID><Version>0</Version><Level>2</Level><Task>0</Task><Opcode>0</Opcode><Keywords>0x8080000000000000</Keywords><TimeCreated SystemTime='2023-01-13T12:36:43.162186500Z'/><EventRecordID>228610</EventRecordID><Correlation/><Execution ProcessID='628' ThreadID='12196'/><Channel>System</Channel><Computer><username></Computer><Security UserID='xxxxxxxx'/></System><EventData><Data Name='param1'>application-specific</Data><Data Name='param2'>Local</Data><Data Name='param3'>Activation</Data><Data Name='param4'>{8BC3F05E-D86B-11D0-A075-00C04FB68820}</Data><Data Name='param5'>{8BC3F05E-D86B-11D0-A075-00C04FB68820}</Data><Data Name='param6'><username></Data><Data Name='param7'>VSP</Data><Data Name='param8'>S-1-5-21-1701108424-2405136955-501842542-1001</Data><Data Name='param9'>LocalHost (Using LRPC)</Data><Data Name='param10'>Unavailable</Data><Data Name='param11'>S-1-15-2-2954403432-4118698401-34248258-1450341969-3067051761-1312270129-654195489</Data></EventData><RenderingInfo Culture='en-US'><Message>The application-specific permission settings do not grant Local Activation permission for the COM Server application with CLSID   {8BC3F05E-D86B-11D0-A075-00C04FB68820}   and APPID   {8BC3F05E-D86B-11D0-A075-00C04FB68820}   to the user <username>\\VSP SID (S-1-5-21-1701108424-2405136955-501842542-1001) from address LocalHost (Using LRPC) running in the application container Unavailable SID (S-1-15-2-2954403432-4118698401-34248258-1450341969-3067051761-1312270129-654195489). This security permission can be modified using the Component Services administrative tool.</Message><Level>Error</Level><Task></Task><Opcode>Info</Opcode><Channel></Channel><Provider>Microsoft-Windows-DistributedCOM</Provider><Keywords><Keyword>Classic</Keyword></Keywords></RenderingInfo></Event>): The application-specific permission settings do not grant Local Activation permission for the COM Server application with CLSID {8BC3F05E-D86B-11D0-A075-00C04FB68820} and APPID {8BC3F05E-D86B-11D0-A075-00C04FB68820} to the user <username>\\VSP SID (S-1-5-21-1701108424-2405136955-501842542-1001) from address LocalHost (Using LRPC) running in the application container Unavailable SID (S-1-15-2-2954403432-4118698401-34248258-1450341969-3067051761-1312270129-654195489). This security permission can be modified using the Component Services administrative tool.

Environment

• Release: DX UIM 20.4, any CU
• Component : UIM NAS
• nas 9.34, 9.36

Cause

• nas defect
• nas crash occurred as soon as the probe started reading the nas.cfg, e.g., ConfCreateRegexpTable log entries
• There were one or more crashes in the nas third-party library -> PCRE lib when it was trying to match the defined regex, for example, "/^((?!PIM99|PST001).)*$/" with the alarm message.

  Additional evidence:
  
  In the nas.cfg, there were a number of SOH entries (Start of Heading viewable in Notepad++) and what looks like corrupt characters.
  Dec 20 14:20:10:398 [17536] 6 nas: confCreateRegexpTable, s = AMSBCKUP, ppch =ðB®‹ª
  Dec 20 14:20:10:398 [17536] 6 nas: confCreateRegexpTable, s = AMSINTL, ppch =`ˆ¯‹ª
  Dec 20 14:20:10:398 [17536] 6 nas: confCreateRegexpTable, flag UseEnhancedRegex = 0, ppch= `ˆ¯‹ª, regexp = ^AMSINTL$
  Dec 20 14:20:10:403 [17536] 6 nas: confCreateRegexpTable, cslRegExpCompile pch =ERCPj, result = à
  ܻ

Resolution

nas-9.36-T3.zip which is attached to this KB Article.