All interfaces down on SSLV

Products

• SSL Visibility Appliance Software

Issue/Introduction

SSLV shows all interfaces down in the middle of traffic processing

 

 

Cause

This is a result of a bug causing workers to stall. Below is what you'll see in the diag logs.

Feb 17 15:13:19 SV3800b-01 ssldata[9296]:  [N] Stall detected in worker 32 of type policy_worker
Feb 17 15:13:19 SV3800b-01 ssldata[9296]:  [N] worker 32 policy_worker: _run T, _stop F, running T, paused F, Dsleep 0, Dspin 0, stall/hthTmo 10000/30000, loop_time 694870359 prog 695059960, now 695060519, stalled T, unhealthy F
Feb 17 15:13:19 SV3800b-01 ssldata[9296]:  [N] Stall detected in worker 34 of type np_worker
Feb 17 15:13:19 SV3800b-01 ssldata[9296]:  [N] worker 34 np_worker: _run T, _stop F, running T, paused F, Dsleep 0, Dspin 0, stall/hthTmo 5000/30000, loop_time 694876844 prog 0, now 695060519, stalled T, unhealthy T
Feb 17 15:13:19 SV3800b-01 ssldata[9296]:  [W] Lockup detected in worker 34 of type np_worker
Feb 17 15:13:19 SV3800b-01 ssldata[9296]:  [N] worker 34 np_worker: _run T, _stop F, running T, paused F, Dsleep 0, Dspin 0, stall/hthTmo 5000/30000, loop_time 694876844 prog 0, now 695060519, stalled T, unhealthy T
Feb 17 15:13:19 SV3800b-01 sslpilot[60759]:  [E] Interfaces UP->DOWN: 9+10+11+12+13+14+15+16
Feb 17 15:13:21 SV3800b-01 ssldata[15916]: [E] Mini Dump Created -->> /var/log/crash/ssldata/405add9c-cf36-451a-589fea8e-fa34a3c0.dmp
Feb 17 15:13:21 SV3800b-01 ssldata[9296]:  [N] Worker 32 of type policy_worker no longer stalled
Feb 17 15:13:21 SV3800b-01 ssldata[9296]:  [N] Worker 34 of type np_worker no longer stalled
Feb 17 15:13:21 SV3800b-01 ssldata[9296]:  [N] Worker 34 of type np_worker recovered

Resolution

This issue is resolved in 4.5.8.1. Upgrade to fix the issue.