Access Gateway JVM Crash in libzip.so (readCEN)

Products

• SITEMINDER

Issue/Introduction

The Symantec Access Gateway JVM experiences an intermittent crash.

Analysis of the fatal error log indicates the crash occurs in the native library libzip.so during a periodic background scan of JAR files.

 

 

Symptoms:

• The hss_error log identifies the problematic frame as: C [libzip.so+0x119fd] readCEN+0x63d

• The stack trace shows the crash occurs under ContainerBackgroundProcessor while checking if JAR files have been modified.

 

Environment

 

• Product: Symantec Access Gateway 12.8 SP7

• OS: RHEL 8.x

• Java: OpenJDK 1.8.x (Specifically applicable to JDK 1.8)

• Component: Embedded Tomcat Server

 

Cause

The crash is triggered by Tomcat’s Automatic Deployment/Reload feature. Every 10 seconds, a background thread scans the classpath to check for file changes.

In JDK 1.8, this check uses native C code (libzip.so). If a JAR file is replaced or modified while the JVM is reading the "Central Directory" (readCEN) of the zip file, it can result in a SIGBUS or a memory fault, crashing the entire JVM.

Note: This issue is not applicable to JDK 11 or higher, as the ZIP handling was rewritten in Java, removing the dependency on the vulnerable native readCEN implementation.

 

Resolution

Option 1: Disable JAR Scanning (Recommended Workaround)

To prevent the JVM from performing these periodic scans and avoid the crash:

1. Navigate to <install_path>/Tomcat/conf/.

2. Open catalina.properties.

3. Add or update the following line: tomcat.util.scan.StandardJarScanFilter.jarsToSkip=*

4. Restart the Access Gateway.

Option 2: Upgrade JRE

Since the crash is a known limitation of the native ZIP implementation in older Java versions, upgrading to JDK 11 will natively resolve the risk of this specific crash.

Option 3: Data Collection for Root Cause Analysis

If you cannot apply the workaround and need to identify which external process is modifying the files:

1. Enable core dumps (e.g., ulimit -c unlimited).

2. If the crash recurs, provide the core file and the hs_err log to Support for analysis.