"Too many open files" error seen in the Spectrum OneClick catalina.out file on Linux

Products

• CA Spectrum

Issue/Introduction

The following error is seen in the $SPECROOT/tomcat/logs/catalina.out file on Linux:

28-Apr-2021 17:17:46.432 SEVERE [http-nio-8080-Acceptor] org.apache.tomcat.util.net.Acceptor.run Socket accept failed
 java.io.IOException: Too many open files
  at sun.nio.ch.ServerSocketChannelImpl.accept0(Native Method)
  at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:422)
  at sun.nio.ch.ServerSocketChannelImpl.accept(ServerSocketChannelImpl.java:250)
  at org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:463)
  at org.apache.tomcat.util.net.NioEndpoint.serverSocketAccept(NioEndpoint.java:73)
  at org.apache.tomcat.util.net.Acceptor.run(Acceptor.java:95)
  at java.lang.Thread.run(Thread.java:748)

Environment

Release : 10.4.* running on Linux

Component : Spectrum OneClick

Resolution

Increase the number of open files in the processd_init.sh file by doing the following on the OneClick system.

1. Log into the system or su to the root user

2. Make a backup of the $SPECROOT/lib/SDPM/processd_init.sh file

3. Edit the $SPECROOT/lib/SDPM/processd_init.sh file. Look for the following entry:

ulimit -c unlimited

Change this to read as follows:

ulimit -n 65536

4. Save the change

5. Stop and then start processd

6. Enter the following command where <PID> is the process id of the processd process:

cat /proc/<PID>/limits

You should now see the following:

Max open files            65536                65536                files

7. Monitor the catalina.out file for the error.