SSP UI fails to load post upgrade

Products

• VMware vDefend Firewall
• VMware vDefend Firewall with Advanced Threat Prevention

Issue/Introduction

Following upgrade of SSP, the SSP User Interface (UI) may become completely inaccessible. Attempting to access the dashboard may experience page loading failures, HTTP 500-series errors, or connection timeouts.

Upon inspecting the underlying issue, core system microservice pods (such as sentinel) stuck in CrashLoopBackOff or Init states. Dependent feature services and background cronjobs may also fail to initialize properly.

Environment

SSP 5.0 and above

Cause

This issue typically occurs due to an incomplete pod initialization sequence following an upgrade, stemming from one or more of the following conditions:

• Missing Schema or Migration Jobs: Critical initialization manifests (such as sentinel-migration-job) are missing, failed, or were manually deleted during upgrade troubleshooting before execution completed.

• Node Desynchronization: Extra or orphaned node instances remain registered in the vCenter while being absent from the platform SSP-I CLI inventory.

• Cascading Microservice Blockages: Core management pods remain blocked in the Init state waiting for database migration jobs to signal completion, preventing dependent background workers, flow managers, and UI components from coming online.

Resolution

Step 1:

Resolve Infrastructure & Node Inventory Mismatches

- Inspect the registered node, machine, and virtual machine resources using the platform CLI:

 

# k get nodes

# kubectl get machines -A

# kubectl get vspheremachines -A

# kubectl get vspherevms -A

- Compare the output against the active nodes inventory in vCenter under SSP resource pool.

- If unmapped control/worker nodes exist only at the vCenter level, safely remove the extra machine instances directly from vCenter.

 

Step 2:

Restore Missing Initialization Jobs

 

- Check and verify the health status of SSP under the SSP-I UI diagnosis page.

- Note down the pod name and verify the same after ssh to SSP-I using sysadmin.

- Check the pods and related jobs status.

# k get pods -A | egrep -vi "completed|running"


# k get jobs -A | grep "failed"

 

- If any job and pod is failing, try to restart the jobs.

 

# k delete job <job name> -n <namespace>

 

If this does not resolve the issue, kindly reach out to the Broadcom support for further assistance.