Collect Logs for Troubleshooting a Disconnected Agent

Products

• Carbon Black App Control (formerly Cb Protection)

Issue/Introduction

To collect logs to troubleshoot a disconnected App Control agent.

Environment

• App Control Agent: All Supported Versions
• Microsoft Windows: All Supported Versions
• macOS: All Supported Versions
• Linux: All Supported Versions

Resolution

Windows

1. On both the disconnected endpoint and the App Control server collect a network trace using the built-in Netsh or Wireshark
   • To collect Netsh, open Command Line as Administrator and run:

     netsh trace start capture=yes tracefile=%userprofile%\Desktop\Agent-Trace.etl persistent=yes

   • Alternately, collect Wireshark capture if it is available
2. In the administrative command prompt, issue the following commands:

   cd "C:\Program Files (x86)\Bit9\Parity Agent"
   dascli status
   
   dascli password GlobalPassword (if no password is available, skip the rest) 
   dascli disconnect
   dascli setconfigprop max_rolling_trace_size_mb=500
   dascli setconfigprop max_rolled_trace_logs_to_keep=4
   dascli debuglevel 7
   dascli nettrace 1
   dascli connect

3. In the dascli status output, note the Server Address and Port, example:

   Current sever: appcontrol.domain.com:41002

   • The Agent will make a DNS call to resolve the Server Address.
   • All commands for connectivity should be attempted against this Server Address
4. Use PowerShell to issue the following commands, substituting the relevant Server Address as returned by the Agent:

   Test-NetConnection -ComputerName <ServerAddress> -Port 41002 -InformationLevel "Detailed"
   Test-NetConnection -ComputerName <ServerAddress> -Port 443 -InformationLevel "Detailed"
   

5. In the administrative command prompt, issue the following commands and screenshot the results:

   netstat -ano | findstr "41002"
   nslookup <ServerAddress>
   ping <ServerAddress>
   tracert <ServerAddress>
   C:\Windows\SysWOW64\netsh.exe winhttp show proxy
   (for XP/Server 2003 use: proxycfg)
   

6. In the administrative command prompt, capture logs and reset Debug Levels:

   dascli capture "C:\Temp\%ComputerName%-DisconnectedLogs.zip"
   dascli password GlobalPassword
   dascli debuglevel 0
   dascli nettrace 0
   dascli setconfigprop max_rolling_trace_size_mb=50
   dascli setconfigprop max_rolled_trace_logs_to_keep=0

7. Stop the Network trace from Command Line (or stop the Wireshark capture):

   netsh trace stop

   • Wait at least 2 minutes for the ETL trace file to finalize
8. Once finished, copy or screenshot the output of dascli status and ALL network commands
9. Zip all files, including the network captures and upload them to the case; please provide an update when done

macOS

1. On the disconnected endpoint use Terminal to issue the following commands to verify the Agent connection status:
   

   cd /opt/bit9/bin
   ./b9cli --status

2. If the Agent is fully running and otherwise healthy: authenticate with the Agent and issue the following commands:

   cd /Applications/Bit9/Tools
   ./b9cli --password 'GlobalCLIPassword'
   ./b9cli --disconnect
   ./b9cli --setconfigprop max_rolling_trace_size_mb=500
   ./b9cli --setconfigprop max_rolled_trace_logs_to_keep=4
   ./b9cli --debuglevel 4
   ./b9cli --nettrace 1
   ./b9cli --connect
   ./b9cli --status
   

3. In the returned output, locate: Server Information > Server and note the address
   • Example: "appserver.domain.com:41002" means the Server Address is appserver.domain.com
4. Set the Debug Levels back to the defaults and collect the logs:

   ./b9cli --debuglevel 0
   ./b9cli --kerneltrace 2
   ./b9cli --nettrace 0
   sudo ./b9cli --capture /var/tmp/DisconnectedAgentLogs.zip
   ./b9cli --setconfigprop max_rolling_trace_size_mb=50
   ./b9cli --setconfigprop max_rolled_trace_logs_to_keep=0

5. Collect the System Logs::

   system_profiler -detailLevel full > ~/Desktop/sysinfo.txt
   

6. Upload all captured logs  and update the existing Support Case.

Linux

1. 1. On the disconnected endpoint use Terminal to issue the following commands to verify the Agent is running and has a Server listed:

      cd /opt/bit9/bin
      ./b9cli --status
      

      • If no Server is listed, the install was missing the server.conf file and the Agent will need uninstalled via Rescue Mode before reinstalling.
      • If the Agent is not running, follow the steps to start it manually.
   2. If the Agent is fully running and otherwise healthy: authenticate with the Agent and issue the following commands:

      ./b9cli --password GlobalCLIPassword
      ./b9cli --disconnect
      ./b9cli --debuglevel 4
      ./b9cli --kerneltrace 4
      ./b9cli --nettrace 1
      ./b9cli --connect
      ./b9cli --healthcheck
      ./b9cli --status
      

   3. In the returned output, locate: Server Information > Server and note the address
      • Example: "appserver.domain.com:41002" means the Server Address is appserver.domain.com
   4. Attempt communication between the endpoint and the Server Address by running the commands:

      ping <SERVERADDRESS>
      nslookup <SERVERADDRESS>
      telnet <SERVERADDRESS> <SERVERPORT>

      Note: If the Telnet utility is unavailable the timeout utility (part of the coreutils package) could be used instead:

      timeout 1 bash -c "</dev/tcp/ServerAddressHere/41002"
      echo $?
      

      An exit status of "0" indicates the Server Address is responding on the port specified (41002).
   5. Set the Debug Levels back to the defaults and collect the logs:

      ./b9cli --debuglevel 0
      ./b9cli --kerneltrace 2
      ./b9cli --nettrace 0
      sudo ./b9cli --capture /var/tmp/DisconnectedAgentLogs.zip

   6. Collect the System Logs::

      sudo tar cvfz /var/tmp/SystemLogs.tgz /v

   7. Collect a 5 minute packet capture:

      sudo tcpdump -i any -w /tmp/PacketCapture.pcap
      
      wait 5 minutes then press CTRL+C to stop the capture

   8. Upload all collected data.