Launcher crashed when running a lot of executions.
book
Article ID: 86488
calendar_today
Updated On:
Products
CA Automic Dollar Universe
Issue/Introduction
Affects Release version(s): 5
Error Message : In Universe logs : ################ uxioserv /u_io_create_thread /000000000 - u_io_create_thread - System error 12 : thread launch type 0 failed (unable to create new thread), thread_number xxx ################
In IO serv logs : ################ [Log File DOLLAR UNIVERSE] - not enough resources, check UXMAXFD environment variables and/or system limits for maximum number of file descriptors/threads/processes per process/user/system wide - system function [fopen] error [EMFILE] - : Too many open file
[u_io_create_thread] - not enough memory - system function [pthread_create] error [ENOMEM] - : Cannot allocate memory ################
Description :Launcher crashed when running a lot of executions.
Cause
Cause type: Configuration Root Cause: The UXMAXFD Dollar Universe variable sets the maximum amount of files descriptors that a process (server, engine , ...) can open.File descriptors limitation for the current shell session especially can be check and modified with ulimit -n.When UXMAXFD is not defined, Dollar Universe can use maximum of 256 file descriptors, which can be quickly too low if you are running a lot of jobs simultaneously. The value is linked to files or network sockets requested by the use of Dollar Universe.- If UXMAXFD is lower or equal to ulimit -n, everything goes fine until Dollar Universe reaches UXMAXFD value, then new actions will wait until enough file descriptors are available.
Ask System administrator if there is no special limitation on your node's host before modifying ulimit -n. When starting, ioserv announce how many FD it is allowed to take in the logs, but this value will only refer to UXMAXFD, not to ulimit -n, even if ulimit -n is lesser (Which would be BAD).
Check OS limits with : ulimit -n cat /proc/sys/fs/file-max
In Dollar Universe 5.x : - Load environment - Stop company - Verify that all Dollar Universe process are correctly stopped. - Edit uxsetenv files in $UXMGR (uxsetenv, uxsetenv_ksh, uxsetenv_csh) - Add UXMAXFD (with a correct value). Example for uxsetenv and uxsetenv_ksh: UXMAXFD=4096 export UXMAXFD Example for uxsetenv_csh: setenv UXMAXFD 4096 - Reload environment - Start company