How to clear Provisioning Server notification queue (inbound synchronization)
Article ID: 20352
Updated On:
Products
Issue/Introduction
In some cases, a notification sent from Provisioning Server to Identity Manager cannot be processed by Identity Manager, due to various reasons. While this notification is in place, it is blocking the queue, preventing from further notifications (updates) from Provisioning to reach and be processed by Identity Manager.
Possible error visible in etanotify.log
- Error in notification processing: Reason: Operation failed. ERROR: IMS was not able to consume the notification successfully.
- DONE: Notifications Processed: 0/100+ [FAILED]
The procedure below describes the procedure of identifying the relevant notification and removing it, so the queue is again functional and able to process subsequent pending notifications.
Environment
All versions of Identity Suite (Identity Manager)
Resolution
Identifying Use case:
Q: First how do we identify whether the notification queue is blocked?
A: By reviewing the latest etanotifyxxxxxxxx-xxxx.log under (..\CA\Identity Manager\Provisioning Server\logs) search for the "Notifications Processed" information
START: Notify Batch Processing
Sending Notification: eTNotifyOpID=1da866bf-3b1f-4a0d-xxxx-4612099ceba1
Event: Delete_Provisioning_Object (eTDYNDirectoryName=<directory name>)
SeqNo: 0000000514
Try sending payload to
https://<im.domain.com>:8443/idm/ETACALLBACK/?env=<env name>
ERROR: No message provided
Error in notification processing: Reason: Operation failed. ERROR: IMS wasnot able to consume the notification successfully.
Originated from: .\EtaNotifyTools.cpp [1102].
DONE: Notifications Processed: 0/100+ [FAILED]
The line above states that this notification processing has failed and that there are > 100 notifications pending in the queue
Removing blocking notification:
Next step would be to connect to the notification DSA (<HOSTNAME>-impd-notify) using an LDAP browser and remove the problematic notification:
Note: Please make sure to have a backup of the DSA in place before making any of the following changes!
1. Connect to the notification DSA - default port 20404 on the Provisioning Server hostname using the user DN eTDSAContainerName=DSAs,eTNamespaceName=CommonObjects,dc=im,dc=etadb together with the password used during installation.
(You could also lower security temporarily by setting min-auth=none; via DSA Management Console for the DSA)
2. From the etanotifyxxxxxxxx-xxxx.log, identify the eTNotifyOpID of the problematic notification. From the example given earlier, we can see ours is
eTNotifyOpID=1da866bf-3b1f-4a0d-xxxx-4612099ceba1
3. Using your LDAP browser session, search for it and once found delete it.
Next you should observe the rest of the pending notifications being processed (by reviewing the updated etanotifyxxxxxxxx-xxxx.log). Should you encounter further "faulty" notifications blocking the queue, steps 1-3 mentioned earlier are to be followed for each one.
Additional Information
If the number of pending notifications is too high and they are of no importance, there is a way of clearing the database of all notifications at once.
Please be aware you will delete all inbound notification so this operation has a potential of data loss if any notification has importance.
This can be done by following these steps:
1. Stop IM
2. Stop the Provisioning Server
3. Stop the notify DSA (<hostname>-impd-notify)
4. Keep a copy of the .db file (<hostname>-impd-notify) in an external location - for backup purposes
5. Execute dxemptydb <hostname>-impd-notify - this re-creates an empty DB
6. Start the notify DSA
7. Start the Provisioning Server
8. Start IM
Feedback
