Rapid service status is down on the NDR Sensor.

Products

• VMware vDefend Firewall
• VMware vDefend Firewall with Advanced Threat Prevention

Issue/Introduction

Rapid service status is in a down state on the Sensor. This means it is having trouble processing file analysis requests from the sniffing pipeline.

Environment

SSP 5.1

NDR Sensor 5.1

Cause

The Rapid service on the NDR Sensor is responsible for analyzing files for potential malware threats. A down state can occur due to the following reasons:

Resource exhaustion: The service has defined memory limits and timeouts for its components. When the file analysis volume is too high, these limits may be exceeded, leading to slowness or instability.

Dependency issues: The Rapid service relies on the sniffing service to receive files for analysis. If the sniffing service is down, Rapid cannot operate properly.

Connectivity problems: Issues with the MPS service on SSP connecting to the cloud can also cause the Rapid service to go down.

Resolution

Rapid Service not starting up could be temporary and has been designed to restore on its own.

If this does not happen after 30 mins, then it might be worth checking the below troubleshooting steps from the NDR Sensor CLI using admin credentials to diagnose and resolve the issue:

 

1. Confirm the service status: Access the NDR Sensor CLI and check the status of the "rapid" service.

ndr-sensor> get service rapid

This will show the current runtime state of the service.

 

2. Restart the service:  A restart can often resolve temporary issues, especially if the service has gotten into a bad state due to a short-term spike in traffic.

ndr-sensor> restart service rapid

After restarting, monitor the service status to see if it returns to a healthy state.

 

3. Verify the dependent services: Ensure the core dependencies are healthy. The "rapid" service cannot function without them.

ndr-sensor> get service sniffing

If the "sniffing" service is down, refer to instructions available in the KB article - "Sensor Sniffing Service status is down". 

 

4. Check system resources: If the service becomes down frequently, it could be a sign that the sensor is undersized for the volume of traffic it is analyzing. Check the overall system CPU and memory usage. If they are consistently high, it may indicate a need for more resources. Refer to Sensor KBs - "Sensor CPU Usage is High" and "Sensor memory usage is high"

 

5. MPS service on SSP not having internet connectivity: MPS feature running on vDefend SSP requires cloud connectivity. Rapid service on the sensor depends on the MPS feature running on vDefend SSP and might stop functioning if MPS service on vDefend SSP cannot connect to the cloud.

Refer: Analyst API Service is unreachable: vDefend SSP Alarm

 

If the issue still persists, then it is advisable to collect the NDR Sensor support bundle (refer documentation for how to collect support bundle) and raise a support ticket.