Understanding Internet Gateway error: An existing connection was forcibly closed by the remote host

Products

• IT Management Suite
• Client Management Suite

Issue/Introduction

The customer noticed the following log entries on his Internet Gateway log:

Exception during receive from [ip:192.0.2.3:35221, id:5cc94901-9b3b-4b2f-a731-74414d3c9950, type: Client, disposed: False] and sending to [ip:192.0.2.2:61932[192.0.2.1:4726], id:01ca5617-f868-4f7c-a43a-f4377d842433, type: Server, disposed: False]: System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
— End of inner exception stack trace —
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error)
at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
at System.Net.Security.SslStreamInternal.<FillBufferAsync>g__InternalFillBufferAsync|38_0[TReadAdapter](TReadAdapter adap, ValueTask`1 task, Int32 min, Int32 initial)
at System.Net.Security.SslStreamInternal.ReadAsyncInternal[TReadAdapter](TReadAdapter adapter, Memory`1 buffer)
at System.Threading.Tasks.TaskToApm.End[TResult](IAsyncResult asyncResult)
at System.Net.Security.SslStream.EndRead(IAsyncResult asyncResult)
at InternetGateway.SslSocket.SafeRead(IAsyncResult res)
-----------------------------------------------------------------------------------------------------
Date: 1/11/2021 10:43:40 AM, Tick Count: 0 (00:00:00), Size: 1.42 KB
Process: InternetGateway (13248), Thread ID: 22, Module: InternetGateway.exe
Priority: 1

We want to understand the different sections of what this error means.

Environment

ITMS 8.5

Resolution

The following sections of the log entry refer to:

Regarding:

Exception during receive from [ip:192.0.2.3:35221, id:5cc94901-9b3b-4b2f-a731-74414d3c9950, type: Client, disposed: False]

Usually, the first portion refers to the Client Machine and the IP Address received by the Internet Gateway. It is usually the external IP Address when the client machine is in CEM mode. As well, in those environments where there is a proxy or network device, that could be the IP Address for that device.

Regarding:

and sending to [ip:192.0.2.2:61932[192.0.2.1:4726], id:01ca5617-f868-4f7c-a43a-f4377d842433, type: Server, disposed: False]

Usually the first IP Address listed is the Internet Gateway IP Address (in this example: 192.0.2.2). Usually the exposed one to the internet.
The second IP Address mentioned (in this example: 192.0.2.1) usually refers to the SMP or a Site Server. 

In this case, when the message says "An existing connection was forcibly closed by the remote host", the connection is closed by the server-side (in this case, the SMP). The exact reason is not 100% clear. It could be that some Client Machines have a revoked / expired certificate. The client machine can connect to the Internet Gateway since revocation lists are not up to date but the SMP is rejecting the connection.

Additional Information

Check for revoked certificates (certificate revocation is usually a Microsoft owned process): 

On the Notification Server:

1. Load MMC and open the Certificate Revocation List sub folder in the Trusted Root Certification Authorities folder. 
2. Backup the CRL for the SMP and Remove the CRL for the SMP server.  

On the Gateway:

1. Browse to "Program Files\Symantec\SMP Internet Gateway\crl", then delete CRL files.
2. Open Internet Gateway user interface, refresh the SMP Server so it can clear out the previous revocations.