Lot of Tasks in Delayed State in System Delivery Engine

Products

• CA Client Automation - IT Client Manager
• CA Client Automation
• CA Client Automation - Software Delivery

Issue/Introduction

In System Delivery Engine there are a lot of Tasks in red with Delayed State.

Environment

Client Automation 14.0, 14.0 SP2 and 14.5 CU1

Cause

This could have multiple causes but one possible cause is when plugin Notification Server (cfnotsrvd) is receiving too many messages.

 

camstat -n | find /I "U-NS"

returns a queue U-NS with a big value for QLEN

 

Example :

Application           proto state  port  Qlen  m/sent  m/recv  retry  disc hold
---------------       ----- ----- ----- ----- ------- ------- ------ ----- ----
U-NS                    tcp   CON  4105   234  603737 8117451      0     0   60

cmengine.exe process are sending a lot of Notification message 1.2.16.2 (SOFTWARE_UPDATED) when it collects Intellisig Software Inventory..
This overloads the U-NS queue and other notifications are taking time to be processed.

 

This is due by a comparison of Software Version with different format (x.y.0.0 and x.y) 

- In Engine logs we could see this :

Engine4_PR|cmEngine        |cCollectSoftwareDete|000000|DETAIL | cCollectSoftwareDetected::ProcessIntellisigSw sw def (1527c705-839a-4832-9118-54d4Bd6a0c89, 10.0) exists
Engine4_PR|cmEngine        |cCollectSoftwareDete|000000|DETAIL | cCollectSoftwareDetected::UpdateSwDef comparing (8, 8) (1527c705-839a-4832-9118-54d4Bd6a0c89, 10.0, 1527c705-839a-4832-9118-54d4Bd6a0c89, 10.0)
Engine4_PR|cmEngine        |cCollectSoftwareDete|000000|DETAIL | cCollectSoftwareDetected::UpdateSwDef language (, )
Engine4_PR|cmEngine        |cCollectSoftwareDete|000000|DETAIL | cCollectSoftwareDetected::UpdateSwDef description (1527c705-839a-4832-9118-54d4Bd6a0c89_10.0.19041.1023_neutral_neutral_cw5n1h2txyewy, 1527c705-839a-4832-9118-54d4Bd6a0c89_10.0.18362.449_neutral_neutral_cw5n1h2txyewy)
Engine4_PR|cmEngine        |cCollectSoftwareDete|000000|DETAIL | cCollectSoftwareDetected::UpdateSwDef sw_version_number (10.0.0.0, 10.0)
Engine4_PR|cmEngine        |cCollectSoftwareDete|000000|DETAIL | cCollectSoftwareDetected::UpdateSwDef bit_support_id (, )
Engine4_PR|cmEngine        |cCollectSoftwareDete|000000|DETAIL | cCollectSoftwareDetected::UpdateSwDef chip_set_id (, )
Engine4_PR|cmEngine        |cCollectSoftwareDete|000000|DETAIL | cCollectSoftwareDetected::UpdateSwDef no change in manufacturer_uuid
Engine4_PR|cmEngine        |cCollectSoftwareDete|000000|DETAIL | cCollectSoftwareDetected::UpdateSwDef set description=?, sw_version_number=?, last_update_date=1631883673

Engine4_PR|CmdPtreImpl_Ado |CmdPtreImpl_Ado.cpp |001113|DETAIL | ExecuteCommandPointer sql: UPDATE ca_software_def SET description=?, sw_version_number=?, last_update_date=1631883673 WHERE sw_def_uuid=?
Engine4_PR|CmdPtreImpl_Ado |CmdPtreImpl_Ado.cpp |001127|DETAIL | ExecuteCommandPointer sqlwparam: UPDATE ca_software_def SET description='1527c705-839a-4832-9118-54d4Bd6a0c89_10.0.18362.449_neutral_neutral_cw5n1h2txyewy', sw_version_number='00010000000000000000', last_update_date=1631883673 WHERE sw_def_uuid=0x6f8ccd1ffd9b1948b8dfb44ba7bdc749

Engine4_PR|cmEngine        |cAmoDomain::SendNoti|000000|DETAIL | Begin
Engine4_PR|cmEngine        |cAmoDomain::SendNoti|000000|DETAIL | Before Send
Engine4_PR|cmEngine        |cAmoDomain::SendNoti|000000|DETAIL | SendNotification1 1.2.16.2, <notification>

Resolution

Open a case at CA Broadcom Technical Support requesting the fix :

 

T5SD196 for Client Automation 14.0

T5IZ449 for Client Automation 14.0 SP2

T5IZ460 for Client Automation 14.5 CU1