RiskFabric Processing job immediately fails

Products

Information Centric Analytics

Issue/Introduction

Information Centric Analytic's (ICA) nightly RiskFabric Processing job or the RiskFabric Intraday Processing job fails almost immediately after starting and an error message similar to the following is saved in the SQL Server Agent history log:

Previous Nightly job Staging is still running [SQLSTATE 42000](Error 50000). The step failed.

The failing job step is Check for Upgrade or previous nightly jobs running (step 1).

Environment

Release : 6.x

Cause

The previous nightly job has not completed, or the SQL Server Agent erroneously indicates the job is still running after a service crash prevented it from recording a job cancellation command. The former can be due to the job hanging or running longer than expected; the latter may be caused by a server reboot or kill command to the service.

Refer to the following knowledgebase (KB) articles to investigate potential causes of the job hanging or running longer than expected:

Note that an under-resourced ICA deployment (relative to the volume of incidents processed) will perform slowly and may be susceptible to environmental issues that degrade or interrupt processing. Environmental factors can include but are not limited to any of the following:

Host OS instability
Multiple databases running on the same server as the RiskFabric relational database, regardless of their distribution across SQL Server instances (for example, running in the default instance or named instances)
Resource contention (for example, disk I/O, or memory consumption)
Network bandwidth limitations between the SQL Server host and data sources, or between SQL Server and the SQL Server Analysis Services (SSAS) host

The SQL Server error log may contain entries that reveal a service interrupt or shutdown coincided with a running processing job. Refer to the Microsoft article View the SQL Server error log in SQL Server Management Studio (SSMS). Executing the following statement in SSMS will display the contents of the latest error log:

EXEC sp_readerrorlog 1;

The following is an example of a service termination entry in the error log caused by a system shutdown command:

10/1/25 7:04    Server    SQL Server is terminating because of a system shutdown. This is an informational message only. No user action is required.

Resolution

Refer to the following KB articles for guidance on investigating and improving processing performance:

For ICA's resource and topology requirements and recommendations, refer to the following sections of the Symantec Information Centric Analytics Administrator Guide: