We cannot filter more than 635 CIs with this API call: https://soi-server/rest/alert?size=635 meaning the error occurs with request size=636.
Error Message in logs :
2021-01-04 21:49:06,043 DEBUG [http-bio-0.0.0.0-7403-exec-11] exceptionmapper.WebApplicationExceptionMapper.toResponse(28) - org.jdom.IllegalDataException: The data "APPMSSQL: Profile ELB_IBSK, Cannot connect to database instance xxxx.yyyy.xxx.xom.lan\AB_CD, 1433: (connection timeout)." is not legal for a JDOM character content: 0x1f is not a legal XML character.
The issue is related to the data of an alert. There may be a special character coming in the message that is creating problem in parsing the alert by using REST API call.
To see that special character, you can copy and paste the SituationMessage and AlertDetail of the alert.
Release : 4.2
Component : Service Operations Insight (SOI) Manager
To fix current issue update the problematic alerts in SOI DB in alert manually as below:
a) Search for the error log as mentioned above in file CA\SOI\SamUI\logs\soiuis-debug.log
b) Find such problematic alarm by using the following query
select * from Alerts where active=1 and SituationMessage like '%Some_part_of_the_Alert_message_from_error_log%'
c) Update the SituationMessage and AlertDetail columns for that alarm by using following query
update alerts set AlertDetail='Remove_Special_Character_and_use_Correct alarm message_here' where alertid=alertid_of_the_problem_alarm
update alerts set SituationMessage='Remove_Special_Character_and_use_Correct alarm message_here' where alertid=alertid_of_the_problem_alarm
d) Stop and Start SOI services one by one
To avoid this issue to occur again the alerts should be corrected in the source product before publishing it to SOI.