Event Forwarder Error "Server is shutting down" When Sending to Splunk

Products

• Carbon Black EDR

Issue/Introduction

When configured to send to Splunk, the event forwarder is able to successfully send some events. Shortly after starting the services, events are no longer getting sent. 

• When sending data to Splunk, the following error message is seen shortly after starting up the services. 

  level=info msg="Error uploading file /var/cb/data/event-forwarder/event-forwarder.2026-02-12T23:48:10.707: HTTP request failed: Error code 503 Service Unavailable\n{\"text\":\"Server is shutting down\",\"code\":23}"

• Disk is filling up. Many files are being stored under /var/cb/data/event-forwarder/ 

Environment

• Carbon Black EDR: All Versions
• Carbon Black Event Forwarder: All Versions

Cause

Firewall between a load balancer and the Splunk servers. 

Resolution

Validate the load balancer is able to reach each Splunk server for ingestion of the data. 

Additional Information

• When one server is unable to respond, the load balancer may be waiting for that server to respond and not continuing to forwarder the data to the other Splunk server instances. 
• Splunk Troubleshooting