Two key alerts went to unknown status. They are still in unknown status. Due to this, a major alert was missed. The same metric grouping works fine in DX Dashboard
SAAS
Regression error.
A long regex of over 10K was involved. The regex had a second capturing group that was the culprit.
The troublesome second group:
Agents\|(S(NNNNN)|Example)\|.*Infrastructure\|Agent:ConnectionStatus
This was breaking the Alert which always had an invalid status and was grayed out.
This was changed to
Agents\|(SNNNNN|Example)\|.*Infrastructure\|Agent:ConnectionStatus
A 2023.9.1 Code was implemented and resolved the issue.