Enterprise Manager failed to start due to Failed to map segment from shared object

Products

CA Application Performance Management Agent (APM / Wily / Introscope) INTROSCOPE

Issue/Introduction

When attempting to start the Enterprise Manager, it quickly fails and goes into an orderly shutdown. In the IntroscopeEnterpriseManager.log file, which is in INFO mode, there is one ERROR line to the effect of:

4/28/17 03:54:04.925 PM EDT [ERROR] [main] [Manager] The EM failed to start. /tmp/librocksdbjni2906454849760766685.so: /tmp/librocksdbjni2906454849760766685.so: failed to map segment from shared object: Operation not permitted

In DEBUG mode there is a much larger stack trace beginning with:

4/28/17 04:00:30.663 PM EDT [DEBUG] [main] [Manager] Error starting up EM

com.wily.introscope.server.enterprise.EnterpriseManagerCannotStartException: An error occurred while trying to start Isengard.

at com.wily.introscope.server.enterprise.EnterpriseServer.doStart(EnterpriseServer.java:418)

at com.wily.util.ALifeCycle.start(ALifeCycle.java:86)

at com.wily.introscope.server.enterprise.EnterpriseServer.<init>(EnterpriseServer.java:324)

at com.wily.introscope.server.enterprise.EnterpriseServer.<init>(EnterpriseServer.java:303)

at com.wily.introscope.server.enterprise.EnterpriseServer.start(EnterpriseServer.java:1595)

at com.wily.introscope.em.internal.Activator.startEM(Activator.java:119)

...

Environment

This particular issue is seen on Linux systems only where the /tmp filesystem is mounted with the "noexec" filesystem option, or is otherwise treated as a non-executable filesystem by the kernel (perhaps by some external security hardening, kernel extension, etc).

Cause

Contrary to the stack trace here, Isengard — which is responsible for low-level networking in the EM — is actually not at fault. What is at fault is the usage of a library known as RocksDB, specifically in the context of the Environment statement above. RocksDB is a third-party library ( http://rocksdb.org ) that was implemented in APM 10.5 for the metadata portion of the SmartStor database where metrics are stored. During EM initialization, the RocksDB library extracts a portion of native code in the form of a shared library (.so file) that it puts into the temporary directory defined by Java system property java.io.tmpdir. Generally this is the /tmp directory on the operating system. RocksDB is then expecting to use and execute the code within that shared library in the same way that any program uses a shared library: the library needs to be opened and mapped such that the bytes from it can be executed.

The issue arises when the operating system prohibits execution of the code within the shared object file (in this example, /tmp/librocksdbjni2906454849760766685.so but yours will likely have a different number which is random). This is the case when underlying filesystem prohibits code execution because it is mounted with the "noexec" flag. This is to say that despite the permissions on the .so file itself, the underlying filesystem (and thus the kernel) prohibits execution of the files contained therein. This is a common security practice in organizations that run hardened systems, especially on shared, world-writable filesystems like /tmp.

Resolution

There are two possible resolutions:

Ensure that the filesystem where /tmp resides is NOT mounted "noexec" and that files can be executed there. A simple test of this is to place an executable file in /tmp and ensure that it can in fact be executed; Or....
Specify a new location for the temporary directory that the EM uses via the java.io.tmpdir Java system property. To achieve this, edit the Enterprise Manager's Introscope_Enterprise_Manager.lax file and update the property lax.nl.java.option.additional to add a define option to specify your desired temp directory path:
-Djava.io.tmpdir=/path/to/your/desired/tmpdir
Of course please make sure that the path you choose is on an executable filesystem.