Primary Hub goes down - MCS log: Failed to load archive list to check package availability

Products

• DX Unified Infrastructure Management (Nimsoft / UIM)

Issue/Introduction

The primary hub becomes unreachable. (IM shows no connection to the local hub) / AC won't connect.

A simple UIM service restart fixes the issue

By verifying netstat -ano output it is noticeable a repeated connection entry from a java process. The PID for this process points to automatic_deployment_engine.

 

MCS log shows messages like:

 

Dec 11 15:59:45:427 [TemplatePackageAvailabilityController[checkPackageAvailability]-30026, mon_config_service] RetryNimRequest.retryOnCommSessionError:124:    Detected session error sending request. Attempting retry on error: Received status (4) on response (for sendRcv) for cmd = 'nametoip' name = 'automated_deployment_engine'
Dec 11 15:59:45:540 [TemplatePackageAvailabilityController[checkPackageAvailability]-30026, mon_config_service] TemplatePackageAvailabilityController.runSyncForUnavailableTemplates:191:    Failed to load archive list to check package availability

Environment

DX UIM 23.4CU2

Cause

In MCS 23.4.* and later a MCS task exists that is managed by a key "check_package_availability". When this key is set to "true" it scans the automated_deployment_engine probe (ADE) for package availability at defined intervals. This mechanism is designed to avoid profiles to get stuck because of package availability inconsistencies.

In large environments, with a high number of templates and custom packages in the archive, this mechanism may cause an excessive consumption of number of ports by the ADE and create the conditions for the above-mentioned issues. 

Resolution

If this happens it is suggested to edit the default interval with which the check_package_availability activity.

• • Steps to edit the interval of check_package_availability:
    • • Open MCS raw configure
      • in the timed section edit check_package_availability_interval from "60" (value in seconds and 60 is the default) to a higher value (Eg.  "1800" - 30 minutes)

Additional Information

Note: if the check_package_availability key is set to "false" it can cause this issue: Unable to create any new device / group MCS profiles: Template disabled