Event partition creation is not being triggered

Products

CA Spectrum DX NetOps

Issue/Introduction

The reporting database is out of sync. The SRM (Spectrum Report Manager) stopped processing the events.

mysql> select * from landscape\G

*************************** 1. row ***************************

landscape_h: 1048576

domain_name: #####

avail_sync_time: 2017-04-01 23:59:53

dev_sync_time: NULL

int_sync_time: NULL

process_if_app_events: 0

spm_sync_time: NULL

alarm_sync_time: 2017-04-01 23:59:40

event_sync_time: 2017-04-01 23:49:46

event_polling: 1

device_polling: 1

1 row in set (0.00 sec)

Environment

Release:
Component:

Cause

The following message errors were found in the Tomcat log file ($SPECROOT/tomcat/logs/stdout.log or catalina.out):

Apr 06, 2017 00:37:39.951 (SRM/LandscapeManager/LandscapeThread_0) (SRM_Events) - (ERROR) - Failed to insert event into event table. SQL Exception code: 1526 SQL Exception message: Table has no partition for value from column_list Last known event time = 1491101993000

Apr 06, 2017 00:37:39.951 (SRM/LandscapeManager/LandscapeThread_0) (SRM_Events) - (ERROR) - event record (mh=0x1233c2, type=0x10801, time=2017-04-02 00:00:01.0)

Apr 06, 2017 00:37:40.871 (SRM/LandscapeManager/LandscapeThread_0) (SRM_Events) - (ERROR) - Database ERROR processing events for domain #####. Will not poll domain

The partitioning handler was not triggered because the Tomcat service is being restarted every 24 hours.

Resolution

Let the Spectrum Tomcat service running more than 24 hours, so the partition handler will be triggered and the new event partition tables will be created.

In the tomcat log file we can see the event tables were created.

abr 11, 2017 12:40:02 AM(INFO)Partition Handling Task started...

abr 11, 2017 12:40:02 AM(INFO)In getMissingPartitions method...

abr 11, 2017 12:40:02 AM(INFO) The last partition name is : EVENT_P91

abr 11, 2017 12:40:02 AM(INFO) We need to create 6 number of partition(s).

abr 11, 2017 12:40:02 AM(INFO) Missing Partition(s) created successfully.

abr 11, 2017 12:40:02 AM(INFO) Updating Partitionlog table.

abr 11, 2017 12:40:02 AM(INFO) Partitionlog table updated with 6 entries.

abr 11, 2017 12:40:02 AM(INFO)In deleteOldPartitions method...

abr 11, 2017 12:40:02 AM(INFO)8 number of partition(s) should be deleted.

abr 11, 2017 12:40:02 AM(INFO) Deletion of older partitions completed successfully.

The Spectrum Tomcat service needs to be recycled to process the events again.

Or you can change when the partition handler is triggered.

1.Navigate to $SPECROOT\tomcat\webapps\spectrum\WEB-INF\repmgr\config\container

2.Edit the Scheduling.xml file.

3.Search for partitionHandlerScheduler bean, change the start delay from 300000 (5 minutes) to 3600000 (1 hour).

<bean id="partitionHandlerScheduler"

class="org.springframework.scheduling.quartz.SimpleTriggerFactoryBean">

<property name="startDelay" value="300000" /> [Change this value to 3600000]

</bean>

4.Save the file and restart the tomcat.

The change will now trigger the partition handler after one hour of tomcat restart instead of default 5 minutes. The repeat interval still holds true with repeat every one day.