Spectrum post_olb_script failure - "script error is 127" (event [0x00010962])

Products

• Spectrum
• Network Observability

Issue/Introduction

After upgrading from 10.3 to 10.4.1, a critical alert appears, "failure during running of the post_olb_script".

The running of the post_olb_script failed after a successful On-Line backup. The processd error is 0, the script error is 127. (event [0x00010962])

The DDMdb database was NOT converted from MyISAM to InnoDB engine since we upgraded from Spectrum 10.2.2 to 10.3.0 and then to 10.4.1.

https://knowledge.broadcom.com/external/article?articleId=138715

Environment

Release : 10.4.1

Component : SPCAEM - Spectrum Alarm & Event Management

Cause

The reason the customer is getting the error while running the post_olb_script is that the script is calling a filename (ddm_backup) that no longer exist in the $SPECROOT/SS/DDM directory:

## Do an online save of the DDM database every Sunday. 
#
# The following is an example of the use of this script.  If you wish to
# enable this functionality, just uncomment the following lines.
#
#day_of_week=`date +%w`
#target_day=0
#
#if [ $day_of_week -eq $target_day ]
#then
    cd $SPECROOT/SS/DDM

    backup_dir=`dirname $ss_db_filename`
    ./ddm_backup $backup_dir/db_$time_string.DDMDb > $backup_dir/ddm_backup.log
    retval=$?
#fi

exit $retval

In Spectrum 10.4.0 onwards, the ddm_backup.exe has been renamed to ddm_backup_legacy.exe.

Resolution

Even after modifying the $SPECROOT/custom/CsScript/post_olb_script file and changing it from calling ddm_backup to ddm_backup_legacy, the post_olb_script is still failing.

If in a bash shell (bash -login), manually launching the ddm_backup_legacy.exe generates the following error:

$ ./ddm_backup_legacy test
    Error: The ddm_save executable does not exist in the current directory.
           Unable to continue.

Then rename the ddm_save_legacy.exe file to ddm_save.exe in the $SPECROOT/SS/DDM/ directory.

This should allow the DDMdb database to be successfully backed up via the post_olb_script.

Additional Information

As Best Practices, we recommend converting the DDMdb from MyISAM to InnoDB engine in the near future.

This is covered in the following section of the Spectrum guide

https://techdocs.broadcom.com/us/en/ca-enterprise-software/it-operations-management/dx-netops/24-3/upgrade-dx-netops/upgrading-dx-netops/upgrade-fault-monitoring/migrating-and-upgrading.html