Process engine not working with bg error: "heart beat is old"

Products

• Clarity PPM On Premise
• Clarity PPM SaaS

Issue/Introduction

Processes are not working, and the following error is shown in the bg-ca.logs

ERROR 2018-07-07 11:30:02,101 [Process Monitor (tenant=clarity)] bpm.engine (clarity:process_adminzzzz__zzzzzzzzzzz:none) ----> Engine: bg-zzzzz heart beat is old. Heartbeat: Sat Jul 07 06:42:43 PDT 2018 Limit: Sat Jul 07 09:30:01 PDT 2018. Engine will be shutdown.. If this is not normal, please fix any problems and restart the process engine. Trying to shudown process enigne. 
java.lang.Exception 
at com.niku.bpm.engine.ProcessMonitor.execute(ProcessMonitor.java:188) 
at com.niku.bpm.engine.ProcessMonitor.run(ProcessMonitor.java:118) 

What does the error mean and how can we resolve the issue?

Environment

Release: All Supported Releases

Cause

• The error means the heart beat is old.
• One of the process pipelines is not responding to the process monitor thread.
• When this happens the "heart beat is old" error is sent when the thread has not responded for two hours and shuts the whole process engine down

 

Resolution

BG is typically going to recover by completely restarting the services

If this didn't happen: 

• Restart BG services 
• SaaS/GCP Customers: Please contact Broadcom support

This could also be a symptom of Bad Multicast network. Implement JDBC Ping instead of Multicast as a workaround. Documentation Link