The DLP detection servers' status is fluctuation between Unknown and Running in the Enforce console.
search cancel

The DLP detection servers' status is fluctuation between Unknown and Running in the Enforce console.

book

Article ID: 173145

calendar_today

Updated On:

Products

Data Loss Prevention Enforce

Issue/Introduction

You are wondering why your detection servers status in the DLP Enforce console System > Servers and Detectors > Overview page is frequently changing from Unknown (red) to Running (green) and back again.

The incident queue number is zero and has not increased for sometime. 

In addition to your production environment you have recently setup a second environment which is your test environment. 

The test environment is a direct copy of your production environment which is running on the same network as your production environment and is used for testing purposes only. 

Both the test Enforce and Oracle Database server are running separately to the production Enforce and Oracle Database server. 

On your detection server you look at the DLP logs and find the following warning repeatedly occurring in the BoxMonitor logs located in the \SymantecDLP\Logs\Debug\ directory: 

Nov 25, 2018 9:19:04 PM com.vontu.communication.transport.ChannelManager handleOperationSuccess
WARNING: Replaced connection for: controller-server and the remote IP for the old connection is: /192.0.2.1. There might be another client connecting to this channel.
Nov 25, 2018 9:19:04 PM com.vontu.communication.dataflow.TransportManager connectionDown
INFO: Connection down for address: controller-server, OPERATION_ERROR
Nov 25, 2018 9:19:04 PM com.vontu.communication.dataflow.TransportManager connectionUp
INFO: Connection up for address: controller-server
Nov 25, 2018 9:19:04 PM com.vontu.communication.transport.TCPAcceptOperation select
INFO: accepted connection from: x.x.x.x:1398
Nov 25, 2018 9:19:04 PM com.vontu.communication.transport.PlainAcceptOperation select
INFO: accept read succeeded: controller-server
Nov 25, 2018 9:19:04 PM com.vontu.communication.transport.SSLAcceptOperation select
INFO: SSLAccept from controller-servercompleted successfully 
session cipher suite: TLS_RSA_WITH_AES_128_CBC_SHA
verified peer name: CN=Vontu
Nov 25, 2018 9:19:04 PM com.vontu.communication.transport.ChannelManager handleOperationSuccess
WARNING: Replaced connection for: controller-server and the remote IP for the old connection is: /192.0.2.2. There might be another client connecting to this channel.
Nov 25, 2018 9:19:04 PM com.vontu.communication.dataflow.TransportManager connectionDown
INFO: Connection down for address: controller-server, OPERATION_ERROR
Nov 25, 2018 9:19:04 PM com.vontu.communication.dataflow.TransportManager connectionUp
INFO: Connection up for address: controller-server
Nov 25, 2018 9:19:04 PM com.vontu.communication.transport.TCPAcceptOperation select
INFO: accepted connection from: x.x.x.x:1123
Nov 25, 2018 9:19:04 PM com.vontu.communication.transport.PlainAcceptOperation select
INFO: accept read succeeded: controller-server
Nov 25, 2018 9:19:04 PM com.vontu.communication.transport.SSLAcceptOperation select
INFO: SSLAccept from controller-servercompleted successfully 
session cipher suite: TLS_RSA_WITH_AES_128_CBC_SHA verified peer name: CN=Vontu

 

The IP address 192.0.2.1 belongs to the production Enforce server and the other 192.0.2.2 belongs to a test development Enforce server. 

This indicates a communication error and it show the two Enforce servers are competing to establish a communication channel with the detection server BoxMonitor service but it is getting confused and tries to revert to the original production IP address.

This warning started appearing in the logs on the date when you setup your test environment.

Cause

Since your test Enforce server is a direct copy of the production Enforce server it believes that it too should be communicating with the production detection servers.

It has all the necessary configuration and certificates to authenticate communication with the production detection servers.

Both environments retain the same certificates for authentication in communications with the detection servers and run over the same communications TCP port 8100 (default port). 

As soon as the Vontu Monitor Controller service starts on the test Enforce server it can successfully communicate with all the detection servers that have been added in the Enforce console.

Resolution

The first step to resolve this issue is to stop and disable Symantec DLP Monitor Controller Service on the test Enforce server. Then restart the  Symantec DLP Detection Server service on the detection servers.

This will restore communications between the production Enforce server and the production detection servers. 

You will now see the incident queue grow again however any incidents generated in the past days since you setup the test environment which interrupted the communications may not be visible or available for incident processing and remediation. 

It is not advisable to have a direct copy of your DLP Production environment running on the same network as your preexisting DLP Production environment.

Ideally you would move the second DLP environment to a separate network.

If you require two DLP environments to be running on the same network, you should: 

  1. If the second Enforce is a direct copy of the existing environment remove all production detection servers from the test Enforce server as they should not be communicating.

  2. Change the TCP port used for communication in the second environment to something else e.g. 8200 to avoid communication conflicts. 

Additional Information

After disabling your test Enforce server and restarting the detection servers you may also need to recycle the production enforce server via the console.

Powered by Wolken Software