Troubleshoot a CCS Query or CCS Data Collection Job that is hanging using CCS Job Trace

Products

• Control Compliance Suite Standards Server
• Control Compliance Suite
• Control Compliance Suite Standards Module

Issue/Introduction

Control Compliance Suite (CCS)

When running a CCS Query job or data collection, and it is not completing and it seems to be hung.

Environment

CCS 12.6.X

CCS 12.7.X

Cause

It could be the scope of the Query or the CCS Job, or it can be just a lot of data that is being collected making it appear that the job is hung.

Or could be a CCS Manager that is having problems.

Resolution

Enable Job Traceability to help identify the CCS Manager(s) doing the collection.  The Job Trace information can show details if the threads for the Manager, and if the threads are obsolete or still active.

Job traceability

Dumping a job trace

There is an option that is available in CCS console’s Jobs view to dump trace information for an executing job. However, the option is not enabled by default.

Perform the steps below to enable Job Trace functionality in the CCS Console:

1. In the CCS Console folder, make a backup of the jobs.monitor.application.xml file in the  <CCS_Console_folder>\Applications\jobs\jobs.monitor.application.xml location.

   1. NOTE: To find the CCS Console folder, open Task Manager à go to SymConsole.exe process à select ‘Open File Location’.  Each user will have their own CCS Console install folder.

2. Edit the "jobs.monitor.application.xml" file and search for keyword ‘Traceability’, and uncommented the specified XML tags below
   NOTE: There are two sections with ‘Traceability’ in the file that will need to be uncommented)

3. Tags are <!-- and -->. Just move the --> tag up to the end of the line with <!-- as shown in the screenshots below

   
   
   
   
   
4. Search for the second ‘Traceability’ section in the file and make the same change to uncomment the section.
   
   
5. Save the file and relaunch the console
   
   
6. The traceability option will then be available under 3 dot menu in the jobs view in the Console as shown below:

The job trace is dumped as an HTML/XML. The required data is present in a hierarchical way i.e. job can be traced from APS to DPS to WorkerProcess or parent job to child job. Traceability to the agent is not covered.

The trace is available only for running jobs\sub-jobs and not for completed jobs\sub-jobs.

Once the option is enabled in the console the trace can be dumped for the selected job or all running jobs. Use the 3 dots dropdown menu from the Jobs workspace to access the "Trace running jobs" option.
If the job is not showing up just select the option to dump all jobs.

Additional Information

NOTE: When in the Job workspace if you press the Alt key on the keyboard it will reveal a traditional file menu at the top of the CCS Console.
You may also find the "Trace running jobs" below under Tasks, see the screenshot below.