Recently we have found that several sudo processes appeared to never complete. Similar sudo commands are successful and the hang appears on less than 1% of similar commands. The parent process was killed, and these are now hung owned by the kernel processID 1
[root@server: /root] ps -ef | grep -i sudo
root 37222 1 0 Jun27 ? 00:00:00 sudo -i -u username ls -rt /xxxxx/outbound_in
root 211021 1 0 Jun25 ? 00:00:00 sudo -i -uusername ls -rt /xxxxx/outbound_in
root 449106 1 0 Jun24 ? 00:00:00 sudo -i -u username ls -rt /xxxxx/inbound
root 835540 1 0 Jul01 ? 00:00:00 sudo -i -u username ls -rt /xxxxx/inbound
root 1044492 1 0 Jun25 ? 00:00:00 sudo -i -u username ls -rt /xxxxx/outbound_in
root 1048884 1 0 Jun25 ? 00:00:00 sudo -p ___SNC_PWD_PROMPT___869b0 ls -l /opt/server/tomcat/webapps/ROOT//WEB-INF/web.xml
root 1357524 1518524 0 04:00 ? 00:00:00 sudo -i -u username chmod 664 /xxxxx/*
Redhats review showed this
crash> foreach sudo vm | awk '/opt/ { print($5) }' | sort | uniq -c
48 /opt/CA/AccessControl/lib/64/libacdki.so
96 /opt/quest/lib64/nss/libnss_vas4.so.2
96 /opt/quest/lib64/security/pam_vas3.so
The futex in itself provides no clue, because it's just pointing to some user address, which is the lock structure.
However it's not possible to know more about that address because it seems in the swap, which is not collected with the memory snapshot, since it's only memory, not the disk.
While the primary cause appears to be the Vintella (VAS) pam libraries the fact that a PAMSC library is in the mix we would want PAMSC to review to see if this may be related to the issue here.
The underlying cause was of why the "sudo -i" command itself hung not identified but removing the PAMSC service from interacting with it is suggested.
While the cause was not identified, it was determined that there was no need for PAMSC to interact with the Linux pam process for this command. To avoid any future interactions, we have suggested to use the following seos.in configuration to stop the seos service from interacting with the pam process.
; This token determines which services(applications) will be bypassed by pam_seos
; for example: ftp
; Default Value: ftp,vsftpd
; bypass_services = service1[,service2,..]
bypass_services = ftp,vsftpd