vDefend SSP Alarm: Node status is degraded or down

Products

• VMware vDefend Firewall
• VMware vDefend Firewall with Advanced Threat Prevention

Issue/Introduction

You are running vDefend SSP 5.0 or later and have encountered an alarm with the description:
"Node {{ .ResourceID }} is not running."

This alarm indicates that a node in the Security Services Platform is either down or degraded and is not functioning as expected.

Environment

vDefend SSP Version: 5.0 and later

Cause

• Resource Exhaustion: High CPU, memory, or disk usage that causes the node to become unresponsive or unstable.
• Kubernetes Node Failures: Issues with Kubernetes node components such as the kubelet, or node being marked as "NotReady" due to insufficient resources, failed pods, or crashed components.

Resolution

For initial troubleshooting, login into SSPI node and execute following commands. 

k get nodes - This command provides a high-level overview of all the nodes in the cluster. It shows details like the node's name, its status (e.g., Ready, NotReady), roles, age, and version of Kubernetes it's running.
k describe node <node-name> - This command gives you a detailed breakdown of a specific node's health and status. 

Refer to the following document for troubleshooting if node is degraded / down. 

https://techdocs.broadcom.com/us/en/vmware-security-load-balancing/vdefend/security-services-platform/5-0/security-services-platform-installer/troubleshooting-sspi/troubleshooting-workload-cluster.html