UMA ebpf prototype, app-container-monitor crashes through runtime error
search cancel

UMA ebpf prototype, app-container-monitor crashes through runtime error

book

Article ID: 380044

calendar_today

Updated On:

Products

DX Application Performance Management DX APM SaaS

Issue/Introduction

Running the ebpf enabled UMA and The app-container-monitor is crashing on one node, for reason unknown 

In the logs I can see the below error

I0926 15:16:57.056232  964410 applicationdataprovider.go:141] namespace otel-demo, pod otel-demo-grafana-bc74c89f5-2bmvr - discovered application of type Go with ppid 1661602, ppid 1660611, inode 27943846
panic: runtime error: index out of range [4] with length 4
       panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x8f00de]


Details of the kubernetes namespace:

$ k get pods -n caapm -o wide
NAME                                              READY   STATUS             RESTARTS           AGE   NODE            NOMINATED NODE   READINESS GATES
apm-probe-autoattach-monitor-7f6654b556-s6csj     1/1     Running            0                  8d    k8s-jm-node2    <none>           <none>
apmia-http-collector-bc69fb578-d8lkk              1/1     Running            0                  8d    k8s-jm-master   <none>           <none>
app-container-monitor-dsnkl                       2/2     Running            0                  8d    k8s-jm-node1    <none>           <none>
app-container-monitor-hqjkf                       2/2     Running            0                  8d    k8s-jm-node2    <none>           <none>
app-container-monitor-wn69z                       1/2     CrashLoopBackOff   1590 (3m21s ago)   8d    k8s-jm-master   <none>           <none>
cluster-performance-prometheus-6c56bcb7f7-9gsf4   1/1     Running            0                  8d    k8s-jm-node2    <none>           <none>
clusterinfo-5546ff5fc9-7r78t                      1/1     Running            0                  8d    k8s-jm-node1    <none>           <none>
container-monitor-6b5549496-4ft7p                 1/1     Running            0                  8d    k8s-jm-master   <none>           <none>

Details on the node that is crashing:

k describe node k8s-jm-master
Name:               k8s-jm-master
Roles:              control-plane,node,worker
..

k get events -n caapm
LAST SEEN   TYPE      REASON    OBJECT                            MESSAGE
5m38s       Warning   BackOff   pod/app-container-monitor-wn69z   Back-off restarting failed container

The events show no reason for it to stop. Only that it is restarting it.

Environment

DX APM UMA 24.7.1.5

Cause

The problem is related to defect # DE617099. To be fixed in the next release 24.11. For an update contact Broadcom Support.

Resolution

None

Additional Information