Time Slicing job failures - troubleshooting / known causes

Products

• Clarity PPM On Premise
• Clarity PPM SaaS

Issue/Introduction

This article provides some tips to aid in troubleshooting time slicing job failures including links to KBs on known causes/solutions.

Resolution

Table of Contents

• Check bg-ca.logs for blobcrack errors
• Related KBs on time slicing job failing
  • Deadlocks
  • Defects Fixed in 15.9.1
  • Defects Fixed in 15.9.2
  • Defects Fixed in 15.9.3
  • Other Job Failures
  • Time Slicing job Stuck / Hanging / Slow
• Additional troubleshooting help

Check bg-ca.logs for blobcrack errors

 

• See KB Accessing the app-ca.log or other Clarity Logs for steps on how to access the bg-ca.logs (and other logs for both On Premise and SaaS customers)
• Debugging configuration: If there are no relevant errors related to time slicing found in the logs, for each server's logging configuration, confirm the following components are set accordingly:

Priority	Name (Component)	Appender
WARN	com.niku	STDOUT
FATAL	org.jgroups	STDOUT
ERROR	org.logicalcobwebs.proxool	STDOUT
ERROR	com.niku.blobcrack	STDOUT
ERROR	com.niku.njs	STDOUT

See KB Setting Logging Levels in Clarity for more details on verifying/setting up the above debugging. 

 

Related KBs on time slicing job failing

 

Deadlocks

The following section covers issues related to deadlocks

General Deadlock issues

• Time Slicing job fails with deadlock errors - troubleshooting steps - For non defect related issues, it's recommended to review this one first as this contains step by step items to check as part of some of our known causes.
• Time Slicing job fails with deadlock ORA-00060: deadlock detected while waiting for resource - for one time occurrence (or very intermittent)
• Time Slicing job deadlock error - due to OWB save
• Oracle deadlock condition causes process error
• Database Deadlock/Slowness post upgrade to 15.9.1 - (DE60813 - Impacts 15.9.1, 15.9.2, 15.9.3 - Under Review)
• Time Slicing deadlocks on multiple tables on PostgreSQL (GCP) (DE62122 - Impacts 15.9.2, 15.9.3 - Under Review)

MS SQL Server Database Specific

• Time Slicing job fails due to deadlock issue on MSSQL

Defects Fixed in 15.9.1

• Time Slicing job is failing due to deadlock errors on prteam intermittently (DE58053 - Impacts 15.8.1, and 15.9, Fixed in 15.9.1)

Defects Fixed in 15.9.2

• Time Slicing Job deadlock failure in GCP/PG environment - (DE58053 - Impacts 15.8.1, 15.9, 15.9.1 - Fixed in 15.9.1.1 and 15.9.2)
• Time Slicing failures post 15.9 Patch 1 (deadlock on fin_benefit_plan_details) (Impacts 15.9 patch 1, 15.9.1 - Fixed in 15.9.1 Patch 1 and 15.9.2)

Defects Fixed in 15.9.3

• Per Period Metrics: 'An Exception occurred...' due to Deadlock with Time Slicing (DE61044 - Impacts 15.9.2. Fixed in 15.9.2 patch 1 and 15.9.3)

Other Job Failures

• Time Slicing Job Fails with ORA-01426 Error 
• Time slicing job fails with ORA-01752 error Post upgrade to Oracle 12 
• Time Slicing Job Fails With Exception: [CA Clarity][Oracle JDBC Driver]Object has been closed
• Time Slicing job fails due to deadlock issue on MSSQL database 
• Time Slicing job fails due to duplicate records being inserted
  • MS SQL Error: [SQLServer]Cannot insert duplicate key row in object 'dbo.PRJ_BLB_SLICES' with unique index 'PRJ_BLB_SLICES_U2'.
        SQL error code: 3621
  • Oracle Error: ORA-00001: unique constraint (CLARITY.PRJ_BLB_SLICES_W_AVL_U1) violated 
• 'Slice value is not valid range' in bg logs -
• Time Slicing job fails with generic error 
• Time Slicing job failing with ORA-01652: unable to extend temp segment by 128 in tablespace TEMP

 

Time Slicing job Stuck / Hanging / Slow

• Time Slicing job appears to be hanging - troubleshooting
• Time Slicing job slow performance - troubleshooting
• Times Slicing Job: Check the Status / Estimated Completion Time
• Scheduled Jobs Not Processed by the Job Scheduler (BG Service Issue)
• Time Slicing job slow on 15.9.1+ due to SQL Curves (Fixed in 15.9.3)
• Time Slicing job running longer at rollover 15.9.1 (Fixed in 15.9.1.1)
• Time Slicing Job running longer on rollover (MSSQL) - (DE61494 - Impacts 15.9.1.1, 15.9.2)
• Time Slicing job very slow during monthly rollover (Oracle)

Additional troubleshooting help

If Broadcom support assistance is needed, the below information can aid in troubleshooting:

1. Do other jobs appear to be running okay?  
2. If On Premise, please provide a copy of the bg-ca.logs and bg-system.logs for the time frame the issue occurred. 
3. Is the Administration > Time Slices page up-to-date or behind? (It should all be caught up by the frequency of the scheduled Time Slicing job.
4. If the job is supposed to run every minute, then the "Last Run Date" should be reflecting the closely to that frequency.)
5. Does the job fail consistently or intermittently? 

Additional Information

See also:

• Time Slicing job appears to be hanging - troubleshooting
• Time Slicing job fails with deadlock errors - troubleshooting
• Time Slicing job slow performance - troubleshooting