UIM Maintenance Probe scheduler is not working

book

Article ID: 221871

calendar_today

Updated On:

Products

CA Unified Infrastructure Management On-Premise (Nimsoft / UIM)

Issue/Introduction

 maintenance_mode probe isn't working at all, under it's log we can see:

 

Aug 10 12:39:08:113 WARN  [attach_socket, com.nimsoft.nimbus.NimServerSession] (120) Callback error, Exception in callback for public void com.nimsoft.monitor.probe.MaintenanceModeProbe.cbGetAllActiveWindowSchedules(com.nimsoft.nimbus.NimSession,int): PreparedStatementCallback; SQL [select mw.window_id, mw.schedule_id, mw.dev_id, mw.start_time, mw.end_time from maintenance_window mw, maintenance_schedule ms where ms.schedule_id = mw.schedule_id and mw.end_time > ? and mw.dev_id is null]; No more data to read from socket; nested exception is java.sql.SQLRecoverableException: No more data to read from socket: PreparedStatementCallback; SQL [select mw.window_id, mw.schedule_id, mw.dev_id, mw.start_time, mw.end_time from maintenance_window mw, maintenance_schedule ms where ms.schedule_id = mw.schedule_id and mw.end_time > ? and mw.dev_id is null]; No more data to read from socket; nested exception is java.sql.SQLRecoverableException: No more data to read from socket

               at com.nimsoft.nimbus.NimServerSession$NimServerSessionThread.performCallback(NimServerSession.java:233)

               at com.nimsoft.nimbus.NimServerSession$NimServerSessionThread.handleMessage(NimServerSession.java:178)

               at com.nimsoft.nimbus.NimServerSession$NimServerSessionThread.run(NimServerSession.java:123)

Caused by: org.springframework.dao.RecoverableDataAccessException: PreparedStatementCallback; SQL [select mw.window_id, mw.schedule_id, mw.dev_id, mw.start_time, mw.end_time from maintenance_window mw, maintenance_schedule ms where ms.schedule_id = mw.schedule_id and mw.end_time > ? and mw.dev_id is null]; No more data to read from socket; nested exception is java.sql.SQLRecoverableException: No more data to read from socket

               at org.springframework.jdbc.support.SQLExceptionSubclassTranslator.doTranslate(SQLExceptionSubclassTranslator.java:98)

               at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:73)

               at org.springframework.jdbc.support.AbstractFallbackSQLExceptionTranslator.translate(AbstractFallbackSQLExceptionTranslator.java:81)

               at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:660)

               at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:695)

               at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:727)

               at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:737)

               at org.springframework.jdbc.core.JdbcTemplate.query(JdbcTemplate.java:792)

               at com.nimsoft.nis.dao.MaintenanceWindowDao.query(MaintenanceWindowDao.java:176)

               at com.nimsoft.nis.dao.MaintenanceWindowDao.getAllActiveTimeWindows(MaintenanceWindowDao.java:160)

               at com.nimsoft.monitor.probe.util.MaintenanceWindowUtil.getAllActiveMaintenanceSchedules(MaintenanceWindowUtil.java:217)

               at com.nimsoft.monitor.probe.util.PDSBuilderForActiveWindows.buildPdsForActiveWindows(PDSBuilderForActiveWindows.java:40)

               at com.nimsoft.monitor.probe.MaintenanceModeProbe.cbGetAllActiveWindowSchedules(MaintenanceModeProbe.java:631)

               at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

               at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

               at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

               at java.lang.reflect.Method.invoke(Method.java:498)

               at com.nimsoft.nimbus.NimServerSession$NimServerSessionThread.performCallback(NimServerSession.java:225)

               ... 2 more

Caused by: java.sql.SQLRecoverableException: No more data to read from socket

               at oracle.jdbc.driver.T4CMAREngineStream.unmarshalUB1(T4CMAREngineStream.java:456)

               at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:397)

               at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:257)

               at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:587)

               at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:225)

               at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:53)

               at oracle.jdbc.driver.T4CPreparedStatement.executeForDescribe(T4CPreparedStatement.java:774)

               at oracle.jdbc.driver.OracleStatement.executeMaybeDescribe(OracleStatement.java:925)

               at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1111)

               at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:4798)

               at oracle.jdbc.driver.OraclePreparedStatement.executeQuery(OraclePreparedStatement.java:4845)

               at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeQuery(OraclePreparedStatementWrapper.java:1501)

               at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:122)

               at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeQuery(DelegatingPreparedStatement.java:122)

               at com.nimsoft.db.NimDBPreparedStatementOracle.executeQuery(NimDBPreparedStatementOracle.java:61)

               at org.springframework.jdbc.core.JdbcTemplate$1.doInPreparedStatement(JdbcTemplate.java:703)

               at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:644)

               ... 16 more

 

Aug 10 12:40:00:405 ERROR [DefaultQuartzScheduler_Worker-7, com.nimsoft.monitor.probe.Notifier] Failed to notify nas /NTRG/PH1/JDCAPETSP2070/ems org.springframework.dao.RecoverableDataAccessException: StatementCallback; SQL [select ms.*,mw.WINDOW_START_TIME,mw.WINDOW_END_TIME, (select rtrim(xmlagg(XMLELEMENT(e,dev_id, ',').EXTRACT('//text()')).GetClobVal(),',') dev_ids from (select cd.dev_id from cm_device cd join maintenance_schedule_members msm on cd.cs_id=msm.cs_id where ms.SCHEDULE_ID = msm.SCHEDULE_ID AND cd.probe_name not like 'app_disco%')) dev_ids, (SELECT rtrim(xmlagg(XMLELEMENT(e,CS_ID||'='||DEV_ID, ',').EXTRACT('//text()')).GetClobVal(),',') Csids_devids FROM (select msm.CS_ID, cd.DEV_ID from cm_device cd join maintenance_schedule_members msm on cd.cs_id = msm.cs_id where ms.SCHEDULE_ID = msm.SCHEDULE_ID AND cd.probe_name not like 'app_disco%')) Csids_devids from MAINTENANCE_SCHEDULE ms left join (SELECT MAX(START_TIME) WINDOW_START_TIME,MAX(END_TIME) WINDOW_END_TIME, SCHEDULE_ID FROM MAINTENANCE_WINDOW group by SCHEDULE_ID) mw ON (mw.SCHEDULE_ID = ms.SCHEDULE_ID)]; No more data to read from socket; nested exception is java.sql.SQLRecoverableException: No more data to read from socket

Aug 10 12:40:00:407 ERROR [DefaultQuartzScheduler_Worker-7, com.nimsoft.monitor.probe.Notifier] Failed to notify nas /NTRG/PH1/JDCAPETSP2070/nas org.springframework.dao.RecoverableDataAccessException: StatementCallback; SQL [select ms.*,mw.WINDOW_START_TIME,mw.WINDOW_END_TIME, (select rtrim(xmlagg(XMLELEMENT(e,dev_id, ',').EXTRACT('//text()')).GetClobVal(),',') dev_ids from (select cd.dev_id from cm_device cd join maintenance_schedule_members msm on cd.cs_id=msm.cs_id where ms.SCHEDULE_ID = msm.SCHEDULE_ID AND cd.probe_name not like 'app_disco%')) dev_ids, (SELECT rtrim(xmlagg(XMLELEMENT(e,CS_ID||'='||DEV_ID, ',').EXTRACT('//text()')).GetClobVal(),',') Csids_devids FROM (select msm.CS_ID, cd.DEV_ID from cm_device cd join maintenance_schedule_members msm on cd.cs_id = msm.cs_id where ms.SCHEDULE_ID = msm.SCHEDULE_ID AND cd.probe_name not like 'app_disco%')) Csids_devids from MAINTENANCE_SCHEDULE ms left join (SELECT MAX(START_TIME) WINDOW_START_TIME,MAX(END_TIME) WINDOW_END_TIME, SCHEDULE_ID FROM MAINTENANCE_WINDOW group by SCHEDULE_ID) mw ON (mw.SCHEDULE_ID = ms.SCHEDULE_ID)]; No more data to read from socket; nested exception is java.sql.SQLRecoverableException: No more data to read from socket

Cause

java memory too low

Environment

Unified Infrastructure Management (Any version)

Maintenance_mode probe

Resolution

-Verify if error is displayed in the maintenance_mode log :ORA-04036: PGA memory used by the instance exceeds PGA_AGGREGATE_LIMIT, if yes, work with Oracle DBA on the Oracle-related issue and fix increasing the Oracle PGA memory, to fix it. 

  • Check the ems and maintenance_mode and spectrumgtw probes, if the java memory min/max is too low so increase them to 2048/4096. And with spectrumgtw at 1024/2048.
  • Then a cold start (Deactivate-Activate) such probes.
  • Verify if no errors whatsoever seen in those probes any longer even at loglevel 5.

 

Additional Information

  • The status of a node in maintenance, e.g., white icon, should not be checked in IM as that is legacy behavior not related to OC for the state of the node.

Powered by Wolken Software