Using Telnet or Powershell to troubleshoot connectivity between CCS components.

book

Article ID: 195968

calendar_today

Updated On:

Products

Control Compliance Suite Control Compliance Suite Standards Server Control Compliance Suite Standards Module

Issue/Introduction

Control Compliance Suite (CCS)

Telnet or Powershell can be used to determine connectivity between CCS components outside of CCS.  This helps if communication is being blocked or misrouted to another server.

Environment

CCS Network Ports:  CCS 12.x Network Ports

Common listening ports for the different CCS components:

All CCS Managers: 5600

All Windows agents: 5601

All UNIX/Linux agents: 5600

Resolution

How use Telnet or Powershell to test connectivity 

It is best to test communication from each server that is running a CCS components.  Example: from ManagerA  to ManagerB, and then from ManagerB to ManagerA.

How to run Telnet:

Start -> Run -> cmd (to open a command prompt.  You might have to open the cmd 'As administrator')

Telnet syntax:
telnet <target_server_hostname> <listening_port>

(NOTE: You can use the hostname, FQDN, or IP of the CCS component you would like to connect to can be used to test)

Example.
telnet mgrccstest 5600 (this example shows an attempt to telnet to the target server MGRCCSTEST using port 5600)

If a blank screen appears with no errors a telnet session has been established.  Press another key to break the connection and return to a command prompt.

How to use Powershell to test connectivity

Start Powershell and run the following command.  In this example you are running this on the CCS manager to test communication to a CCS agent, but it could also be used to test communication from an agent to a manager, etc.

Syntax:
Test-NetConnection -Port <Port_number> -ComputerName <agent_IP> -InformationLevel Detailed

Manager listens on port 5600
Windows agent listens on port 5601
Unix\Linux agent listens on port 5600

Example:

PS C:\Users\ebsvc> Test-NetConnection -Port 5601 -ComputerName 10.32.111.237 -InformationLevel Detailed
WARNING: TCP connect to 10.32.184.237:5601 failed


ComputerName            : 10.32.184.237
RemoteAddress           : 10.32.184.237
RemotePort              : 5601
NameResolutionResults   : 10.32.184.237
MatchingIPsecRules      :
NetworkIsolationContext : Internet
IsAdmin                 : False
InterfaceAlias          : Ethernet
SourceAddress           : 10.32.201.85
NetRoute (NextHop)      : 10.32.200.1
PingSucceeded           : True
PingReplyDetails (RTT)  : 1 ms
TcpTestSucceeded        : False

 

If the TcpTestSucceeded in the Powershell results shows False, then it was not able to connect to that port on that agent, and you need to determine what is blocking it.  

If the TcpTestSucceeded in the Powershell results shows True, then check the esmd.log (Unix\Linux) or the esmagent.log (Windows) on the agent to see if the agent registers the connection.  See the 'How to verify that Telnet or Powershell connected to the target server' section below.

 

How to verify that Telnet or Powershell connected to the target server

Just running Telnet or Powershell and connecting to a server is only part of the test.  If the DNS is incorrect, or if there are firewalls between the originating server and the target server, it can seem to connect but it might not be the correct server.  Checking the logs of the target server can confirm that the telnet did indeed connect to the target server is required.

Confirm connectivity by checking the CCS logs on the target server:

For example if you were checking connectivity from a CCS agent to a CCS manager, perform the Telnet section above on the agent to see if it can telnet to the manager.  To verify it connected to the manager, open the C:\Program Files (x86)\Symantec\CCS\Reporting and Analytics\ESM\system\<hostname>\esmmanager.log on the manager.  Scroll to the bottom and see if you see a generic connection attempt like the following:

[04960] [2020/07/23 16:55:21:015] [CRITICAL] Scheduler: connection handshaking failed.
csp__server_handshake: error reading version from client
buffer is not large enough (data length = 3338, buffer size = 2064)

Note the date/time in the entry above on the target server, and compare that to when you ran the telnet command on the originating server.  If it matches then it did connect to the target server correctly.

Log location for the CCS agents: (For checking connectivity issues from CCS manager to a CCS agent)

  • Windows agent: \Program Files (x86)\Symantec\Enterprise Security Manager\ESM\system\<hostname>\esmagent.log
  • UNIX/Linux agent: /esm/bin/<hostname>/esmd.log

 

Note about Firewalls:

Firewalls between the servers may seem to allow the connectivity to be successful, but the CCS components will still not be able to function correctly.  If you are getting the following error, this indicates that the two CCS components are able to connect, but firewall(s) (including the local firewall on the target server) may be impacting the packets and CCS is not able to validate the connection.

[06428] [2020/07/24 17:32:40:294] [INFO] TCP server: 2020/07/24 <IP>: Connect from <IP_removed>

[06428] [2020/07/24 17:32:40:294] [ERROR] TCP server: common port - failed to read initial few bytes from the socket with error 10054

[06428] [2020/07/24 17:32:40:294] [ERROR] TCP server: common port - The CCS manager failed to validate the inbound connection. Ignoring the connection.

If you see the above error in the esmmanager.log file, you typically have firewall(s) impacting the packets.  Contact your network administrator to allow the required ports for CCS to work correctly.

Additional Information

On UNIX\Linux agents, the IPTables (or NFTables on RHEL 8) are often overlooked and can block connection to the CCS agent.  For troubleshooting, disable IPTables and then see if the manager is able to connect to the CCS agent.  If it does, enable IPTables and configure it to allow connectivity to the CCS agent listening on 5600, and also 5599 for agent upgrades.