DX OI - Maintenance Windows Troubleshooting

Products

• DX Operational Intelligence

Issue/Introduction

The following is a high-list of techniques and suggestions to employ when troubleshooting issues with OI Maintenance Windows

Environment

DX OI 23.3 and higher

Resolution

Troubleshooting steps:

1) Logs to check for possible errors:

            maintenance-service pod

• ../maintenance-service/logs/maintenance-service.log

 

2. Alarms not being marked under maintenance

a) Check for error in logs

[main-SendThread(zookeeper:2181)] ERROR org.apache.zookeeper.client.StaticHostProvider - Unable to resolve address: zookeeper:2181

java.net.UnknownHostException: zookeeper

at java.net.InetAddress.getAllByName0(InetAddress.java:1281)

b) java.net.UnknownHostException: jarvis-kafka: Name or service not known

at java.net.Inet6AddressImpl.lookupAllHostAddr(Native Method) ~[na:1.8.0_322]

For alarms to be marked under maintenance kafka, zookepeer and normalizedalarmservice pods must be up and running. 

 

3) Entities remains under maintenance even after the window duration ends

Duplicate the erroneous schedule and create a window of 1 min duration, set repeat= DOES NOT REPEAT, all the entities would be marked out of maintenance  as soon as the new schedule window ends.

Additional Information

https://knowledge.broadcom.com/external/article/190815/aiops-troubleshooting-common-issues-and.html