Reaper is crashing with 'not enough storage' errors

Products

• CA Application Delivery Analysis MTP (NetQoS / ADA)
• CA Infrastructure Performance

Issue/Introduction

Description:

ISSUE

Reaper keeps crashing and will not process data.

DETAILS

In the Reaper logs, you see the following messages:

********** RealtimeReaper **********

19:51:01 Severity 1 - RealtimeReaper starting up
19:51:01 Severity 1 - The service will perform a scheduled restart at 05/15/2011 00:00:00
19:51:01 Severity 2 - hr_test failure 0x8007000e Not enough storage is available to complete this operation.
at .\permpool.cpp 107
19:51:01 Severity 2 - b_test failure at .\nqWinNQUtils.cpp 420
19:51:01 Severity 2 - Not enough storage is available to process this command.
19:51:01 Severity 2 - hr_test failure 0x80004005 Unspecified error at .\nqWinNQUtils.cpp 420
19:51:01 Severity 2 - hr_test failure 0x8007000e Not enough storage is available to complete this operation.
at d:\builds\ra\current\include\nqmemutils.inl 119
19:51:01 Severity 2 - hr_test failure 0x8007000e Not enough storage is available to complete this operation.
at d:\builds\ra\current\include\nqHashTable.h 524
19:51:01 Severity 2 - hr_test failure 0x8007000e Not enough storage is available to complete this operation.
at d:\builds\ra\current\include\nqHashTable.h 847
19:51:01 Severity 2 - hr_test failure 0x8007000e Not enough storage is available to complete this operation.
at d:\builds\ra\current\netflow\nfrtreaper\nfrtreaper.inl 961
19:51:01 Severity 2 - hr_test failure 0x8007000e Not enough storage is available to complete this operation.
at .\permpool.cpp 107

Solution:

This is due to AS data collection being turned on and loading too much into memory. You can turn off AS data collection or change the limits.

To turn off AS data collection

1. Remote desktop to the Harvester
2. In a command prompt, run the following
   
   mysql -P3308 harvester
   update parameter_descriptions set defaultvalue='N' where parameter='enableASDataCollection';
   
   
3. Restart the NetQoS Reaper service

To change the limits

Go to the registry and change the limits. The numbers below are half of the default limits. If it's still too much for the Harvester, halve them again.

• HKEY_LOCAL_MACHINE\SOFTWARE\NetQoS\RealtimeReaper\FifteenMinuteMainLimits
  
  
  • TopBinAsCount = 50 (decimal)
  
  
• TopBinAsNextHopCount = 10 (decimal)

• HKEY_LOCAL_MACHINE\SOFTWARE\NetQoS\RealtimeReaper\OneMinuteMainLimits
  
  
  • TopBinAsCount = 50 (decimal)
  
  
• TopBinAsNextHopCount = 10 (decimal)

Restart the NetQoS Reaper service

Reaper patch

There are some scenarios in which the customer wants to have AS data collection enabled and reaper still crashes even after reducing the reaper limits to the lowest levels. Development released a reaper patch to address such scenarios, please use it just if changing the above limits fails to avoid the reaper to crash.

Patch and instructions to apply it can be downloaded from the link below:

https://ftp.broadcom.com/user/downloads/pub/netqos/product_patches/RA/9.0/nfrtreaper.zip

Environment

Release: RAIB1H99000-9.1-Network Flow Analysis-Interface Bundle-Hardware
Component: