When attempting to start a windows agent, the agent does not start and the agent logs shows something like:
10:25:11.267 AgentService: AwE-5103
10:25:11.268 AgentService: .AxOptions: NoErrorMsgProperties=false
ErrorMsg: AwE-5103 network socket error (3/21/25 10:25 AM)
Details: Network socket error
javax.net.ssl.SSLException: java.net.SocketException: Connection reset
...
Suppressed: java.net.SocketException: Connection reset by peer: socket write error
...
Caused by: java.net.SocketException: Connection reset
What we see is the agent reaching out to the awcomm process on the RMIServer machine and getting info back:
10:25:11.65 AgentService: .AwComm: awcomm connect true 1011
10:25:11.65 AgentService: .AwComm: Get connect info for: C R:WINAGENT AgentService
10:25:11.65 AgentService: .AwComm: sending message to awcomm 22C R:WINAGENT AgentService*
10:25:11.80 AgentService: .AwComm: lookup 6001 Y 10.10.10.1
10:25:11.81 AgentService: .AgentService: createSocket: rmiserver.example.com:6001 From agentserver/10.10.10.2:6002 encrypt true
10:25:11.81 AgentService: .AgentService: using sun ssl
The next thing that happens is that we try to create a socket and connect with an ssl handshake:
10:25:11.147 AgentService: .SocketManager: TLSv1.2 SunJSSE version 1.8
10:25:11.150 AgentService: .KeyManager: ---alias: automic
10:25:11.151 AgentService: .KeyManager: ---KeyMgr: CN=CN, OU=support, O=O, L=Location, ST=UP, C=UP
10:25:11.207 AgentService: .AgentService: createSocket Socket[addr=rmiserver.example.com/10.10.10.1,port=6001,localport=6002]
10:25:11.208 AgentService: .AgentSocketManager: addRmiSocket Socket[addr=rmiserver.example.com/10.10.10.1,port=6001,localport=6002]
10:25:11.208 AgentService: .B: Socket[addr=rmiserver.example.com/10.10.10.1,port=6001,localport=6002]
10:25:11.209 AgentService: .AgentSocketManager: sendHandshake Socket[addr=rmiserver.example.com/10.10.10.1,port=6001,localport=6002]
And this is where the agent sees the connection get reset.
From the RMI Server side, we see at the same time that the RMI Server accepts a connection from the agent:
11:25:11.211 SM:accept: .AxRmiServer$3: accept connection Socket[addr=/10.10.10.2,port=6002,localport=6001]
Fri Mar 21 11:25:11 EDT 2025 accept connection Socket[addr=/10.10.10.2,port=6002,localport=6001]
and then when it goes back for the handshake, it gets a connection reset as well:
11:25:11.279 SM:validate: .AxRmiServer$3$1: reading handshake Socket[addr=/10.10.10.2,port=6002,localport=6001]
11:25:11.279 SM:validate: .B: Socket[addr=/10.10.10.2,port=6002,localport=6001]
11:25:11.283 SM:validate: AwE-5102
ErrorMsg: AwE-5102 Agent error (3/21/25 11:25 AM)
Details: Socket[addr=/10.10.10.2,port=6002,localport=6001]
javax.net.ssl.SSLException: java.net.SocketException: Connection reset
...
11:25:11.284 SM:validate: .AxRmiServer$1: null ErrorMsg: AwE-5102 Agent error (3/21/25 11:25 AM)
Details: java.net.SocketException: Connection reset
Socket[addr=/10.10.10.2,port=6002,localport=6001]
This issue was caused by a firewall rule that was causing the problem where 6001 was blocked. Once the network team opened it again, the agent was able to connect as expected.
Things that can be checked:
Use a telnet command from the RMIServer machine to the agent server on port 6002. Attempt to start the agent and then from the RMIServer machine, run the command:
telnet 10.10.10.2 6002
You'll also want to work with your network team to trace the traffic going from the agent to the rmiserver and back when attempting to start the agent. They can check to see what is causing the connection reset error.