CCS collects data for half of the asset scope and stays in hanging state

Products

• Control Compliance Suite Windows

Issue/Introduction

• Windows agent-less data collection jobs are routed to a site with two CCS Managers in data collector roles.
• The routing rule used is: "Available CCS Manager in site …"
• CCS collects data for half of the asset scope and the job either stays in hanging state or times out if the timeout value was set in job configuration.
• The status of remaining assets is 'scanning' however no progress is being made.
• When the same job is then launched manually, it will again, only collect for half of the asset scope.
• The job is split between two Managers as expected, but always only one of them can collect data.
• When the routing rule is changed to point to the 'active' Manager, data collection succeeds. 
• When the routing rule is set to the hanging Manager, the job remains in 'routing assets' state and will not proceed further.

No errors, the job either times out or needs to be aborted manually.

Environment

CCS 12.0 in a distributed setup, agent-less data collection, the job scope is 400 Windows assets.

Cause

Issue related to CCS Manager in Load Balancer role.

Resolution

Restarting CCS Manager and DPS services on Managers in Data Collector roles does not help.
Restarting same services on Load Balancer (which at the same is the AppServer) will solve the problem for a short time.

1. Make sure no jobs are currently running.
2. Take a screenshot of the current Load Balancer settings.
3. Navigate to Settings -> Deployment View, un-register the Load Balancer and wait until it disappears from the current view.
4. Re-register the Load Balancer, restore all previous settings.
5. Run full Health&Status job, wait until it completes and make sure the Load Balancer displays healthy status (the question mark changes to a 'tick' symbol).
6. There might be old system jobs active in the background preventing the Health&Status job from running - restarting the Application Server service will clear them all.
7. Once the Load Balancer is back up and running, run the data collection job to verify if the issue is resolved.