java.lang.IllegalArgumentException: Request header is too large
Article ID: 262101
Updated On: 10-10-2023
Products
Issue/Introduction
Sporadically and without the possibility of simulating the error or recovering the conditions that generate it, the Tomcat service crashes writing the following error in the nohup.log file:
Jan 26, 2023 9:00:58 AM org.apache.coyote.http11.Http11Processor serviceINFO: Error parsing HTTP request header Note: further occurrences of HTTP request parsing errors will be logged at DEBUG level.java.lang.IllegalArgumentException: Request header is too large at org.apache.coyote.http11.Http11InputBuffer.fill(Http11InputBuffer.java:781) at org.apache.coyote.http11.Http11InputBuffer.parseHeader(Http11InputBuffer.java:942) at org.apache.coyote.http11.Http11InputBuffer.parseHeaders(Http11InputBuffer.java:593) at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:284) at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65) at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:888) at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1597) at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61) at java.lang.Thread.run(Thread.java:748)## A fatal error has been detected by the Java Runtime Environment:## SIGSEGV (0xb) at pc=0x00007f547e11726e, pid=1032696, tid=0x00007f541b2dc700## JRE version: OpenJDK Runtime Environment (8.0_302-b08) (build 1.8.0_302-b08)# Java VM: OpenJDK 64-Bit Server VM (25.302-b08 mixed mode linux-amd64 compressed oops)# Problematic frame:# V [libjvm.so+0x70726e] jni_GetStringUTFChars+0x6e## Failed to write core dump. Core dumps have been disabled. To enable core dumping, try "ulimit -c unlimited" before starting Java again## An error report file with more information is saved as:# /tmp/data_sps00/hs_err_pid****.log## If you would like to submit a bug report, please visit:# https://github.com/adoptium/adoptium-support/issues#
The result is the Apache frontend web-server starts responding with HTTP 503 code (Service Unavailable) to all the requests.
In the Apache access log, at the same date/time, there is simple request for a favicon.ico:
10.105.64.1 - - [26/Jan/2023:09:00:58 +0100] "GET /favicon.ico HTTP/1.1" 503 29910.105.64.1 - - [26/Jan/2023:09:02:28 +0100] "-" 408 -Environment
Release : 12.8.05
Cause
Customer was trying to post the payload with 8kb of headers to Access Gateway UI url with the packet size setting of worker.ajp13.max_packet_size=65536 in the server.conf file works fine.
With the same setting with 8.1kb of header size of payload is posted on Access
Gateway UI using curl command is crashing the Java process of Access Gateway of Tomcat.
Observations
Customer is trying to post the 8.1 kb payload directly onto the Tomcat.
Troubleshooting's performed
Requested customer to use the Apache component to proxy to the Tomcat hosted federation webservices with 8kb and 8.1kb of payload is successful and no Java process crash is observed.
Payload with 8kb Post Request is successful via Access Gateway's Apache proxy.
Payload with 8.1KB is successful via Access Gateway Apache proxy.
Resolution
Resolution / Conclusion
Customer to post the pay load of the headers to tomcat via the Access Gateway proxy component for handling huge payload sizes and to avoid the crashes of the Access Gateway Tomcat services.
Feedback
