MC Error: "Replication for historical Statistics Monitoring data will terminate once the limit is reached. Check connectivity between nodes."

Products

• Management Center
• MC-S400-20
• Management Center - VA

Issue/Introduction

On Management Center (MC) you are seeing warnings like:

WARNING: Replication lag storage limit is at 79%. Replication for historical Statistics Monitoring data will terminate once the limit is reached. Check connectivity between nodes

 

 

You can verify if replication has stopped on the MC via the CLI command:

# failover view

Failover:

        Status: Healthy (3 second replication delay); Warning: error replicating historical Statistics Monitoring data.

Cause

The disk space allocated for replication on Management Center (MC) is nearing it's limit.

The "Replication lag storage limit" is nearly or at 100% causing replication to terminate.

Resolution

Once replication has stopped you will need to take action before the replication of monitoring statistics will begin again.

Disable failover and re-enable it to get the statistics monitoring working again.  Troubleshoot any connectivity issues between the devices to remedy replication issues from recurring.

1. Disable Failover on the primary and secondary partners by logging in and executing the following commands from  configure mode, under failover:
   
(config-failover)# disable
   
   
2. Verify that failover is disabled with:

(config-failover)# view

 

See: Disable MC Failover for more information.

 

See our technical documentation to configure and re-enable failover:  Configure MC Failover and failover.

 

Once reconfigured, monitor replication and troubleshoot any connectivity issues.

Additional Information

Delays in replication may occur due to networking or resource issues.

Reducing the number of records kept on the MC could reduce resources consumed. 

To do change these values on MC via the CLI, in configuration mode, under statistics-monitoring, you can view current settings via: 

 

(config-statistics-monitoring)# view

 

For example, reducing the number of per-minute and per-hour statistics to 1 and 90 days respectively could save on resources.

 

You would make these changes via:

 

(config-statistics-monitoring)# set-per-minute-lifetime 1

  ok

(config-statistics-monitoring)# set-per-hour-lifetime 90

  ok

 

Verify your changes with:

 

(config-statistics-monitoring)# view