One of the clarity servers throws an error "server refused to start a shell/command". The IT/platform team identified that the Clarity OS user had reached the maximum process limit on the Linux OS, causing the error where the server refused to start a shell or command thread. As per the Red Hat vendor's suggestion from previous similar scenarios, they have updated the resource limit values for the user.
Post that we can access the OS account in that server, however except for the beacon services rest of all the services are in the stopped position and the application team has restarted the services. Still, the NSA and App services are stopping automatically and the server application and NSA URLs are not accessible.
nsa-system.log:
Clarity 16.0.0
The pthread create failed situation usually indicates the Linux kernel lacked the necessary resources to create another thread, or it exceeded the system-imposed limit on the total number of threads in a process.
Post correction of the pthread limits on the Linux kernel take note of the clarity services deployed on this system and remove and deploy them back. For example: