Identity Suite Virtual Appliance : [ERROR] notify file usage: 250MB/250MB (100%)

Products

• CA Identity Manager
• CA Identity Governance
• CA Identity Portal
• CA Identity Suite
• CA Directory
• CA Directory

Issue/Introduction

Identity Suite Virtual Appliance during startup reports the following error:

Provisioning Server   192.168.0.30  [OK] im_ps is running
                                                         [OK] co file usage: 100MB/250MB (4%)
                                                         [OK] inc file usage: 120MB/250MB (48%)
                                                         [OK] main file usage: 200MB/250MB (80%)
                                                         [ERROR] notify file usage: 250MB/250MB (100%)

Notification DSA alarm log reports the following:
** ALARM **: File space exhausted

IM Provisioning Server (IMPS) can no longer store notifications and becomes unresponsive.

Environment

Release : 14.x, 14.x CPX

Component : Identity Suite vApp

Cause

Initial cause for this is inability of IMPS to send notifications to Identity Manager server (IMS)
This can happen because IMPS cannot establish a connection to IMS.
Another reason is that some notification cannot be processed by IMS, and blocks the notification queue.
Notifications that IMPS continue to store in the notification DSA eventually exhaust DSA space.

Another possibility is that notification DSA size was not chosen big enough to satisfy IM activities.

Resolution

This knowledge base article:
How to clear Provisioning Server notification queue (inbound synchronization)
explains how to deal with a notification that is blocking the queue.
However, if DSA file space is exhausted, it is not possible to remove the blocking notification using LDAP browser as explained in the above article.

There are two possibilities to fix this problem:

1. Increase the DSA file size

   1. SSH to the vApp machine as the 'config' user

   2. Use 'dsa' user:
      su - dsa

   3. Stop the DSA:
      dxserver stop <DSA_name>

   4. DSA_name should be like ca-prov-srv-01-impd-notify

   5. Backup the DSA data:

   6. Check if there's enough free disk space on your machine

   7. Modify the $DXHOME/config/servers/<DSA_name>.dxi file:
      find "set dxgrid-db-size = 250;" in the file and increase it to whatever value you need. The value is in megabytes. Recommended Size = 2000

   8. Extend the DSA db file:
      dxextenddb <DSA_name>

   9. Start the DSA:
      dxserver start <DSA_name>

2. Discard the notification data
   The below steps are a last resort and a support case should be opened first to confirm if these steps should be performed.
   Steps to perform:
   1. SSH to the vApp machine as the 'config' user
   2. Use 'dsa' user:
      su - dsa
   3. Stop the DSA:
      dxserver stop <DSA_name>
      DSA_name should be like ca-prov-srv-01-impd-notify
   4. Backup the DSA data:
      dxdumpdb -f <LDIF_file_name> <DSA_Name>
   5. Empty the DSA:
      dxemptydb <DSA_name>
   6. Start the DSA:
      dxserver start <DSA_name>