NetOps Portal web server services fail to start post upgrade

Products

• CA Performance Management
• Network Observability

Issue/Introduction

After an upgrade to Performance Center services won't start.

Performance Center services fail to start.

No errors are seen, the start command returns to the command line simply stating it failed to start the service.

DX NetOps Performance Management Portal web server services fail to stay running after an upgrade. They appear to start but fail to remain running.

Environment

All supported DX NetOps Performance Management Portal web server releases

Cause

Incorrectly configured wrapper.java.maxmemory value in the wrapper.conf for the affected service(s).

This can be seen for any of the four Performance Center services.

Resolution

When this problem is observed, the normal service logs don't write messages and the command line start doesn't reveal a specific error. We'll use the Event Manager service as an example.

When we go to the (default path) /opt/CA/PerformanceCenter/EM/bin we see files named hs_error_pid<PID#>.log. The PID represents the PID the service was assigned by the OS before it failed to start. A new one is written with each failed start attempt.

Reviewing the logs we see these messages at the start of the file.

# There is insufficient memory for the Java Runtime Environment to continue.

# Native memory allocation (mmap) failed to map 649193545728 bytes for committing reserved memory.

 

When that is seen, review the (default path) /opt/CA/PerformanceCenter/EM/conf/wrapper.log and the wrapper.java.maxmemory setting value.

If the value shows as if it was written twice, it will cause the service restart to fail.

The example in this case was the value seen in the file was:

• wrapper.java.maxmemory=39863986

It should be:

• wrapper.java.maxmemory=3986

Once the file was edited to use the correct value, the service restarted without error.