Linux Agent: "Verify the user running Nolio agent has privileges to impersonate other users" error
Article ID: 37292
Updated On:
Products
Issue/Introduction
This article is specific to running actions on Linux. If you are receiving the message above while using the impersonation feature on Windows then review Windows Agent: "Verify the user running Nolio agent has privileges to impersonate other users" error .
This article is related to the User Impersonation feature that allows an action to run as another user. By default, with no additional configuration on the agent machine it will attempt to SSH to 127.0.0.1 as the user specified for the action. This can be found by opening the details of the action and going to the Settings/Properties tab. The ROC will show you the username and password field without any additional steps. In ASAP click the "Set Credentials" button to see/change the details.
To use sudo or su, install of SSH, please refer to the product documentation for details on how to properly configure the agent machine. The documentation can be found here: Run Processes Under Different Users – Unix/Linux .
The error/scenario covered by this article is seen while running an action configured to run as a specific user on a Linux/Unix servers. When running the action it returns:
Error occurred during action execution: Unable to run action for user myImpersonateUserId. Verify the user running Nolio agent has privileges to impersonate other users, and that user myImpersonateUserId has permissions on Nolio installation folder.
Six of the eight problems/causes (1-4, 6-8) can be identified by log messages in various logs. To see if you're experiencing any of these problems you can perform the following regular expression search on all the logs in the logs folder. To do this:
- cd to the <nolioagent home/installation>/logs folder.
- Run the command: grep -E "error handling remote process connection|Connection timed out|no tty present|NolioAgent: Permission denied|you must have a tty to run sudo|Resources temporarily unavailable|ActionsRunner.sh: Permission denied" *
- Match the results found with the problem/cause described below to see if it matches and then determine if the solution is appropriate.
Environment
Release Automation: All Versions
Release Automation Agent on Linux
Cause
There are a few scenarios that will generate this message when running it on Linux/Unix. This article will go through known conditions for which this error is returned - one by one. For each one of these issues the standard nolio_all message is seen. Each condition described below will highlight additional log messages you can use to isolate the cause. See additional information for standards used throughout this article.
Briefly, the types of problems we've seen and have solutions for are as follows:
- Timeout while communicating with a sub-process that has established a connection with the parent process.
- Timeout while waiting for the sub-process to connect with the parent process.
- The noliouser does not have access to execute processes on behalf of myImpersonateUserId
- The myImpersonateUserId does not have r-x access to <NolioAgentInstallDir>/jre folder/files and sub folder/files
- The myImpersonateUserId does not have access to any folders/files except <NolioAgentInstallDir>/jre
- The noliouser does not have "Defaults:noliouser !requiretty" in sudoers
- The noliouser sudoers can impersonate myImpersonateUserId but it is not configured appropriately.
- The myImpersonateUserId does not have appropriate resources.
- The myImpersonateUserId does not have appropriate permissions.
- The myImpersonateUserId does not have r-x access to <NolioAgentInstallDir>/ directory
Resolution
The types of problems outlined above are discussed in detail below. Each type/cause has a unique solution which is given immediately after the details of the problem and how to confirm if that problem applies to what you are seeing.
1. Timeout while communicating with a sub-process that has established a connection with the parent process
If this is the cause for the standard user interface error then the following message is typically seen inside of the nolio_all.log:
2016-02-01 10:53:41,937 [ProcessInvoker-Server-0] DEBUG (com.nolio.platform.shared.datamodel.execution.remote.ProcessesInvoker:485) - Handling remote process request...
2016-02-01 10:53:42,045 [ProcessInvoker-Server-0] INFO (com.nolio.platform.shared.datamodel.execution.remote.ProcessesInvoker:495) - new user process connected: myImpersonateUserId
2016-02-01 10:53:42,045 [ProcessInvoker-Server-0] ERROR (com.nolio.platform.shared.datamodel.execution.remote.ProcessesInvoker:510) - error handling remote process connection for user myImpersonateUserId
java.lang.IllegalStateException: cannot find process for user myImpersonateUserId
at com.nolio.platform.shared.datamodel.execution.remote.ProcessesInvoker.handleClient(ProcessesInvoker.java:498)
at com.nolio.platform.shared.datamodel.execution.remote.ProcessesInvoker.run(ProcessesInvoker.java:461)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2016-02-01 10:53:41,940 [main] DEBUG (root:186) - [myImpersonateUserId] connect successfully. sending username myImpersonateUserId
2016-02-01 10:54:41,914 [Thread-0] INFO (root:394) - [myImpersonateUserId] process of user myImpersonateUserId is up
2016-02-01 10:55:17,352 [HealthMonitor] DEBUG
Also, the following messages are typically logged to the <NolioAgentInstallDir>/logs/myImpersonateUserId_output.log:
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/LISAReleaseAutomationAgent/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/LISAReleaseAutomationAgent/actionslib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]NolioSocketAppender is not ready yet!
10:53:41,775 [main] DEBUG (ProcessesConfig.java:89) - remote process execution properties:{cmd.to.execute=./SudoActionsRunner.sh, max.processes=5, core.agent.instance.action.threads=10, max.process.action.threads=5, server.timeout=20000, ssh.port=22, grant.permissions=false, max.idle.time.in.seconds=900, process.timeout=20000}[NolioSocketAppender is not initialized yet] [null] remote process execution properties:{cmd.to.execute=./SudoActionsRunner.sh, max.processes=5, core.agent.instance.action.threads=10, max.process.action.threads=5, server.timeout=20000, ssh.port=22, grant.permissions=false, max.idle.time.in.seconds=900, process.timeout=20000}
10:53:41,812 [main] INFO (ProcessesConfig.java:128) - ProcessInvoker properties:processCreationTimeout-20000, serverInitializationTimeout-20000, grantPermissions-false, coreAgentInstanceActionThreads-10, maxAgentInstanceActionThreads-40, maxRemoteProcesses-5, maxRemoteProcessActionThreads-5, maxIdleTimeInSeconds-900, cmdToExe-./SudoActionsRunner.sh, sshPort-22[NolioSocketAppender is not initialized yet] [null] ProcessInvoker properties:processCreationTimeout-20000, serverInitializationTimeout-20000, grantPermissions-false, coreAgentInstanceActionThreads-10, maxAgentInstanceActionThreads-40, maxRemoteProcesses-5, maxRemoteProcessActionThreads-5, maxIdleTimeInSeconds-900, cmdToExe-./SudoActionsRunner.sh, sshPort-22ActionRunner is running
10:53:41,847 [main] INFO (ActionsRunner.java:137) - changing log configuration[NolioSocketAppender is not initialized yet] [null] changing log configuration
10:53:41,940 [main] DEBUG (ActionsRunner.java:186) - connect successfully. sending username myImpersonateUserId
10:54:41,914 [Thread-0] INFO (ActionsRunner.java:394) - process of user myImpersonateUserId is up
Note:
The "connect successfully. sending username myImpersonateUserId" message seen in the myImpersonateUserId_output.log is confirmation that the sub-process has established a connection with the parent process. The timeout occurring here is while the parent process sends the sub-process instructions on what to do.
The message "cannot find process for user myImpersonateUserId" in the nolio_all.log file is the message that ties this cause to its respective solution.
Resolution:
Increase the <NolioAgentInstallDir>/conf/processes.properties value for "process.timeout". By default it is 20000 (20 seconds).
2. Timeout while waiting for the sub-process to connect with the parent process.
If this is the cause of the standard user interface error then the following messages will be found in the nolio_all.log file:
2018-08-13 13:05:58,521 [main] ERROR (root:195) - [myImpersonateUserId] error occurred
java.net.ConnectException: Connection timed out
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.<init>(Unknown Source)
at java.net.Socket.<init>(Unknown Source)
at com.nolio.platform.shared.datamodel.execution.remote.ActionsRunner.init(ActionsRunner.java:183)
at com.nolio.platform.shared.datamodel.execution.remote.ActionsRunner.main(ActionsRunner.java:107)
This is seen after:
a. The exception that accompanies the standard user interface error. Example:
2018-08-13 13:05:55,351 [job-3735684-jobServer-3735797-6:Run as myImpersonateUserId(P8633694000.F8633698000.E8633699000):Run Command Line] ERROR (com.nolio.platform.shared.datamodel.Action:119) - Exception caught: com.nolio.platform.shared.datamodel.execution.remote.ProcessCreationFailedException: Unable to run action for user myImpersonateUserId. Verify the user running Nolio agent has privileges to impersonate other users, and that user myImpersonateUserId has permissions on Nolio installation folder.
b. A message that may or may not appear regarding the process being up. Example:
2018-08-13 13:05:55,521 [Thread-0] INFO (root:394) - [ldgadm] process of user myImpersonateUserId is up
The message indicating that the process is up is a little misleading. The process does start. But a successful connection between the sub-process and agent process involves the two connections. One connection/port used for the parent process to send instructions. Another connection/port used for the parent to read stdout/stderr generated by the commands run by the sub-process. This port is referred to as the "logger port". The problem reported to technical support was related to a timeout with the logger port. This generated the following additional message in the nolio_all.log file:
2018-08-13 13:05:58,531 [Thread-11] ERROR (com.nolio.platform.shared.datamodel.execution.remote.ConsolidationLogger:98) - Error while handling remote logging
java.io.EOFException
at java.io.ObjectInputStream$BlockDataInputStream.peekByte(Unknown Source)
at java.io.ObjectInputStream.readObject0(Unknown Source)
at java.io.ObjectInputStream.readObject(Unknown Source)
at com.nolio.platform.shared.datamodel.execution.remote.ConsolidationLogger$ClientLoggerHolder.run(ConsolidationLogger.java:87)
at java.lang.Thread.run(Unknown Source)
The myImpersonateUserId_output.log looks pretty standard with the following exception - the "process of user myImpersonateUserId is up" message is seeing after the time out (in this case we had already increased it to 60 seconds):
13:04:55,516 [main] INFO (ActionsRunner.java:137) - changing log configuration[NolioSocketAppender is not initialized yet] [null] changing log configuration
13:05:55,521 [Thread-0] INFO (ActionsRunner.java:394) - process of user myImpersonateUserId is up
13:05:58,521 [main] ERROR (ActionsRunner.java:195) - error occurred
java.net.ConnectException: Connection timed out
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.<init>(Unknown Source)
at java.net.Socket.<init>(Unknown Source)
at com.nolio.platform.shared.datamodel.execution.remote.ActionsRunner.init(ActionsRunner.java:183)
at com.nolio.platform.shared.datamodel.execution.remote.ActionsRunner.main(ActionsRunner.java:107)
The problem was found to be related to a invalid DNS entry for the servername. This was detected by running "netstat -aonp | grep <pid of sub-process>" and seeing something like the following:
tcp 0 1 . ::ffff:192.168.0.12:33258 ::ffff:192.168.2.12:36758 SYN_SENT <pid of sub process>/./jre/bin/Nol <etc.>
In this case the local address (192.168.0.12) is correct address and the foreign address (192.168.2.12) was invalid. When doing an nslookup against the servername it was found to point to the invalid address. The address is being used by the parent process for its end of the logger port connection.
Resolution:
- Update DNS. After DNS is updated you will need to stop the agent, flush dns (and possibly arp cache) and then restart the agent; or
- Update /etc/hosts with an entry for the correct IP address and this machines hostname (short and fully qualified hostname).
3. Noliouser does not have access to execute processes on behalf of myImpersonateUserId
If this is the cause of the standard user interface error then the following message is usually seen in the myImpersonateUserId_output.log:
sudo: no tty present and no askpass program specified
You can usually confirm this by logging into myAgentMachine as noliouser and run:
sudo -u mymyImpersonateUserId echo test
Good result output:
test
Bad result:
Sorry, user noliouser is not allowed to execute '/bin/echo test' as myImpersonateUserId on myAgentMachine.
Resolution:
Add one (only one) of the following lines to the /etc/sudoers file:
- noliouser ALL=(myImpersonateUserId) NOPASSWD:ALL
- noliouser ALL=(ALL) NOPASSWD:ALL
- noliouser ALL=(myImpersonateUserId) ALL
- noliouser ALL=(ALL) ALL
4. The myImpersonateUserId does not have r-x access to <NolioAgentInstallDir>/jre folder/files and sub folder/files
When the myImpersonateUserId doesn't have r-x access to <NolioAgentInstallDir>/jre folder/files and sub folder/files the following message may be logged to the myImpersonateUserId_output.log
./ActionsRunner.sh: line 14: ./jre/bin/NolioAgent: Permission denied
Resolution:
chmod -R 755 <NolioAgentInstallDir>/jre
5. The myImpersonateUserId does not have access to any folders/files except <NolioAgentInstallDir>/jre
When the myImpersonateUserId does not have access to the files/folders (except <NolioAgentInstallDir>/jre) then the myImpersonateUserId_output.log gets created but is blank.
Resolution:
chmod -R 755 <NolioAgentInstallDir>
6. The noliouser does not have "Defaults:noliouser !requiretty" in sudoers
The following message can be seen in the <NolioAgentInstallDir>/logs/myImpersonateUserId_output.log:
sudo: sorry, you must have a tty to run sudo
Resolution:
Add the following line to your /etc/sudoers file:
Defaults:noliouser !requiretty
7. The noliouser sudoers can impersonate myImpersonateUserId but it is not configured appropriately
The following message can be seen in the <NolioAgentInstallDir>/logs/myImpersonateUserId_output.log:
sudo: no tty present and no askpass program specified
Resolution:
Add the following line to your /etc/sudoers file:
noliouser ALL = (myImpersonateUserId) /<NolioAgentInstallDir>/ActionsRunner.sh
or
noliouser ALL = (myImpersonateUserId) NOPASSWD:/<NolioAgentInstallDir>/ActionsRunner.sh
8. The myImpersonateUserId does not have appropriate resources.
The following message can be seen in the <NolioAgentInstallDir>/logs/nolio_all.log:
No privileges.
And the following message can also be seen in the <NolioAgentInstallDir>/logs/myImpersonateUserId_output.log
.../<NolioAgentInstallDir>/ActionsRunner.sh: fork: retry: Resources temporarily unavailable...
Resolution:
Compare the output of the ulimit command (run as both noliouser and myImpersonateId). Make sure that the limits for myImpersonateId are greater than or equal to the limits set for noliouser.
9. The myImpersonateUserId does not have appropriate permissions.
This error is not actually accompanied with the usual "Verify the user running Nolio agent has privileges to impersonate other users, and that user myImpersonateUserId has permissions on Nolio installation folder." message. However, it is a problem related to the setup on the agent machine when impersonating other users.
The following message can be seen in the nolio_all.log:
[main] DEBUG (root:186) - [myImpersonateUserId] connect successfully. sending username <myImpersonateUserId>
[main] ERROR (root:243) - [myImpersonateUserId] Cannot find class that came from main agent process. maybe this process did not removed after actions distribution
java.lang.ClassNotFoundException: com.nolio.platform.shared.executables.actions.exec.RunCommandLine
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
and:
ERROR (com.nolio.platform.shared.datamodel.execution.remote.ConsolidationLogger:98) - Error while handling remote logging
java.io.EOFException
at java.io.ObjectInputStream$BlockDataInputStream.peekByte(ObjectInputStream.java:2950)
Resolution:
chmod -R 755 <NolioAgentInstallDir>
10. The myImpersonateUserId does not have r-x access to <NolioAgentInstallDir>/ directory
When the myImpersonateUserId doesn't have r-x access to <NolioAgentInstallDir> directory the following message may be logged to the myImpersonateUserId_output.log
./ActionsRunner.sh: Permission denied
Resolution:
chmod 755 <NolioAgentInstallDir>
Additional Information
Throughout this document it assumes the following:
- The userid that owns the NolioAgent process is: noliouser
- The userid used in the set credentials section is: myImpersonateUserId
- The agent machine name where you are trying to run an action as another user is referred to by: myAgentMachine
- "<NolioAgentInstallDir>" refers to the root directory where the Nolio Agent was installed (on the Nolio Agent machine where the actions are being run).
- The "standard nolio_all message" refers to this message found in the <NolioAgentInstallDir>/logs/nolio_all.log:
2016-02-01 10:53:39,713 [job-32769-jobServer-32769-6:Run Command Line(P12696000.F12700000.E12701000):Run Command Line] ERROR (com.nolio.platform.shared.datamodel.Action:119) - Exception caught: com.nolio.platform.shared.datamodel.execution.remote.ProcessCreationFailedException: Unable to run action for user myImpersonateUserId. Verify the user running Nolio agent has privileges to impersonate other users, and that user myImpersonateUserId has permissions on Nolio installation folder.
com.nolio.platform.shared.datamodel.execution.remote.ProcessCreationFailedException: Unable to run action for user myImpersonateUserId. Verify the user running Nolio agent has privileges to impersonate other users, and that user myImpersonateUserId has permissions on Nolio installation folder.
at com.nolio.platform.shared.datamodel.execution.remote.ProcessesInvoker.createProcessIfNeeded(ProcessesInvoker.java:313)
at com.nolio.platform.shared.datamodel.execution.remote.ProcessesInvoker.runExecutableOnRemoteProcess(ProcessesInvoker.java:252)
at com.nolio.platform.shared.datamodel.ActionExecutionState.remoteProcessExecution(ActionExecutionState.java:286)
at com.nolio.platform.shared.datamodel.ActionExecutionState.access$400(ActionExecutionState.java:39)
at com.nolio.platform.shared.datamodel.ActionExecutionState$2.execAction(ActionExecutionState.java:109)
at com.nolio.platform.shared.datamodel.ActionExecutionState.exec(ActionExecutionState.java:343)
at com.nolio.platform.shared.datamodel.Action.run(Action.java:227)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at com.nolio.platform.shared.flowcontrol.LocalFlowController$JobBoundThreadFactory$1.run(LocalFlowController.java:788)
at java.lang.Thread.run(Unknown Source)
Caused by: java.util.concurrent.TimeoutException: [myImpersonateUserId] Process creation breach the timeout of 20000 milliseconds
at com.nolio.platform.shared.datamodel.execution.remote.ProcessesInvoker.waitForClient(ProcessesInvoker.java:407)
at com.nolio.platform.shared.datamodel.execution.remote.ProcessesInvoker.createProcessIfNeeded(ProcessesInvoker.java:297)
... 13 more - The "standard user interface error" refers to the message seen in ASAP or ROC stating:
Error occurred during action execution: Unable to run action for user myImpersonateUserId. Verify the user running Nolio agent has privileges to impersonate other users, and that user myImpersonateUserId has permissions on Nolio installation folder.
Troubleshooting:
If a support case is necessary to troubleshoot the issue then the following information usually helps to resolve the problem in a timely manner:
- A copy of the nolioagent_home/logs folder
- the nolioagent.dir file captured by running the following command on the agent: ls -laR <nolioagent_home> > nolioagent.dir
- a copy of the following files:
- conf/processes.properties
- The file/script referenced by the conf/processes.properties -> cmd.to.execute
For example: In this example the conf/processes.properties file's cmd.to.execute points to /opt/nolio/agent/customActionsRunner.sh: cmd.to.execute=/opt/nolio/agent/customActionsRunner.sh
Therefore, we'd want a copy of the /opt/nolio/agent/customActionsRunner.sh file.
- output of the following command while logged in as the owner of the agent process: sudo -l
Feedback
