What is the meaning of the alarm: Insert bulk failed due to a schema change of the target table

Products

• DX Unified Infrastructure Management (Nimsoft / UIM)

Issue/Introduction

What is the Meaning of the following data_engine alarms-> "Insert bulk failed due to a schema change of the target table."

"Bulk load data was expected but not sent. The batch will be terminated."

Environment

• Any version of the data_engine probe through 9.X
• UIM 9.x,20.x

Cause

These alarms are generated by the ODBC driver while the data_engine probe attempts creating a rowset during a time when the table in question is being maintained by the data maintenance/indexing process.

These alarms are considered benign.

Resolution

The data_engine may generate an alarm that says "Insert bulk failed due to a schema change of the target table." This is most likely to happen during the housekeeping procedures which, among other things, are reindexing the tables at the same time the data_engine is bulk-inserting new records into the tables.

For example, the bulk insert contains a row for a insert on a locked object (being reindexed). The data_engine detects this, and saves all of the existing bulk insert packages. It then reinitializes the connection to the database (to make sure there isn't a problem), then inserts the saved bulk packages, and continues processing the data. So no data are being lost in this process.

The data_engine may generated an alarm that says ""Bulk load data was expected but not sent. The batch will be terminated."  This alarm is coming from the ODBC driver that the probe uses when attempting to create a rowset which is blocked for similar reasons. 

When the data_engine generates either of these alarms, the commit fails and the data_engine restores the records back to the unprocessed queue. These records will be processed in the next cycle.

This is behavior by design, and has been implemented to catch the several hundred error conditions which can be generated from the ADO/Native Client layer, if something goes wrong

This alarm can also occur even if automatic reindexing is disabled in the data_engine probe's configuration if partitioning is enabled.  The data_engine maintenance job creates new partitions and re-indexing will be performed during partition creation.

By default, these alarms are issued with severity = critical

There is an option to set the alarm_severity flag in the raw configure module of the data_engine to raise these alarms with a different alarm level if that's desirable. But the drawback is of course that you might get into a situation where the ADO/Native Client layer generates a generic error message which you might want to give attention. Unfortunately we're unable to control the messages provided by these layers, so we just present them when they come.