EDR Server: Start hangs at Coreservices
search cancel

EDR Server: Start hangs at Coreservices

book

Article ID: 290639

calendar_today

Updated On:

Products

Carbon Black EDR (formerly Cb Response)

Issue/Introduction

  • Bad file descriptor message in /var/log/cb/coreservices/coreservices.log
    • File "/usr/share/cb/virtualenv/lib64/python3.8/site-packages/gevent/_socket3.py", line 66, in _dummy
          raise OSError(EBADF, 'Bad file descriptor')
      OSError: [Errno 9] Bad file descriptor
  • Other errors in /var/log/cb/coreservices/coreservices.log stack point to FileNotFoundError
    • Traceback (most recent call last):
        File "/usr/lib64/python3.8/logging/handlers.py", line 855, in _connect_unixsocket
          self.socket.connect(address)
        File "/usr/share/cb/virtualenv/lib64/python3.8/site-packages/gevent/_socket3.py", line 421, in connect
          raise error(result, strerror(result))
      FileNotFoundError: [Errno 2] No such file or directory
  • Coreservices service repeatedly stopping and starting in process list with different PIDs

Environment

  • EDR Server: All versions
  • RHEL 6.10

Cause

One of the name servers in resolv.conf is not accessible. 

Resolution

  1. Edit /etc/resolv.conf
  2. Remove any name servers not available or having performance issues
  3. Restart services

Additional Information

  • nslookup of name servers can be a quick check to confirm reachability, but if the name server has performance issues, it can still cause startup failures. 
  • One of the python packages used by EDR tests that name servers can be resolved during startup. If one fails, start up can fail in various ways on different services.