NetOps Portal Device Manager Service Crashes or Fails to Start

Products

• Network Observability
• CA Performance Management

Issue/Introduction

When checking the service status or reviewing logs, the following indicators may be present:

• The command systemctl status caperfcenter_devicemanager.service shows the service as failed (Result: exit-code).
• Java error reports (e.g., hs_err_pid*.log) or standard output logs show a pthread_create failure: [warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.
• /optCA/PerformanceCenter/DM/logs/DMService.log messages indicating the system failed to start a native thread, example:
  
  

  INFO   | jvm 5    | YYYY/MM/DD HH:MM:SS | # An error report file with more information is saved as:

  INFO   | jvm 5    | YYYY/MM/DD HH:MM:SS | # /opt/CA/PerformanceCenter/DM/bin/hs_err_pid<PID>.log

  INFO   | jvm 5    | YYYY/MM/DD HH:MM:SS | [#######.###s][warning][os,thread] Failed to start thread "Unknown thread" - pthread_create failed (EAGAIN) for attributes: stacksize: 1024k, guardsize: 4k, detached.

  INFO   | jvm 5    | YYYY/MM/DD HH:MM:SS | [#######.###s][warning][os,thread] Failed to start the native thread for java.lang.Thread "HttpClient-56197-SelectorManager"

  INFO   | jvm 5    | YYYY/MM/DD HH:MM:SS | [thread ######## also had an error]

  INFO   | jvm 5    | YYYY/MM/DD HH:MM:SS | #

Environment

The pthread_create failed (EAGAIN) error typically indicates that the process has reached a system-imposed resource limit, such as the maximum number of open files or the maximum number of processes/threads.

Resolution

Increase the system limit for open files to 65535 by following these steps:

1. Log in to the server as the root user or a user with sudo privileges.
2. Edit the /etc/security/limits.conf file.
3. Add or update the following lines to set the open files limit to 65535:
   text

    

   * soft nofile 65535
   * hard nofile 65535

4. Verify the current limits for the service user (e.g., perfmon) by running: ulimit -a
5. Restart the Device Manager service: systemctl restart caperfcenter_devicemanager.service

For detailed information on preparing the environment, refer to the official documentation.