Error: GC overhead limit exceeded when REST API in use with AdminUI
Article ID: 239765
Updated On:
Products
Issue/Introduction
Running script with REST API calls to the AdminUI like this:
#!/bin/bash
echo "Token :";
echo $myToken;
for i in `seq -w 1000`; do myToken=`curl -k -u siteminder:<password> -X POST https://adminui.example.com:8443/ca/api/sso/services/login/v1/token | grep sessionkey | gawk -F: '{print $2}' | grep "\"" | gawk -F\" '{print $2}'`; curl -k -H "Authorization: Bearer $myToken" -H "Content-Type: application/json; charset=UTF-8" -X POST --data @agent$i.json https://adminui.example.com:8443/ca/api/sso/services/policy/v1/SmAgents -v; done;
The AdminUI reports the following error:
server.log :
2022-03-01 10:00:52,248 ERROR [io.undertow.request] (default task-1) UT005023: Exception handling request to /ca/api/sso/services/policy/v1/SmAgents:java.lang.OutOfMemoryError: GC overhead limit exceeded
at com.ca.siteminder.rpc.type.InStream.nextTree(Unknown Source) at com.ca.siteminder.rpc.type.InStream.getChildren(Unknown Source) at com.ca.siteminder.rpc.type.InStream.nextTree(Unknown Source) at com.ca.siteminder.rpc.type.InStream.getChildren(Unknown Source) at com.ca.siteminder.rpc.type.InStream.nextTree(Unknown Source) at com.ca.siteminder.rpc.type.InStream.getChildren(Unknown Source) at com.ca.siteminder.rpc.type.InStream.nextTree(Unknown Source) at com.ca.siteminder.rpc.type.InStream.getChildren(Unknown Source) at com.ca.siteminder.rpc.type.InStream.nextTree(Unknown Source)
The AdminUI also quickly uses more memory:
20340 root 20 0 4483684 1.5g 34196 S 19.0 9.5 2:51.12 java 20340 root 20 0 4517608 1.7g 34200 S 22.0 11.2 2:57.56 java 20340 root 20 0 4536112 1.8g 34200 S 30.7 11.8 3:00.65 java 20340 root 20 0 4557700 1.9g 34200 S 23.0 12.3 3:03.53 java 20340 root 20 0 4618352 2.0g 34204 S 138.0 12.7 3:22.80 java
and the batch script hangs here:
} % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 655 100 655 0 0 5131 0 --:--:-- --:--:-- --:--:-- 5117 #* About to connect() to adminui.example.com port 8443 (#0) #* Trying 10.0.0.1... #* Connected to adminui.example.com (10.0.0.1) port 8443 (#0) #* Initializing NSS with certpath: sql:/etc/pki/nssdb #* skipping SSL peer certificate verification #* SSL connection using TLS_DHE_RSA_WITH_AES_256_CBC_SHA #* Server certificate: #* subject: CN=adminui.example.com #* start date: Feb 23 10:42:39 2022 GMT #* expire date: Feb 21 10:42:39 2032 GMT #* common name: adminui.example.com #* issuer: CN=adminui.example.com > POST /ca/api/sso/services/policy/v1/SmAgents HTTP/1.1 > User-Agent: curl/7.29.0 > Host: adminui.example.com:8443 > Accept: */* > Authorization: Bearer <value> > Content-Type: application/json; charset=UTF-8 > Content-Length: 403 > #* upload completely sent off: 403 out of 403 bytes < HTTP/1.1 500 Internal Server Error < Connection: keep-alive < Content-Type: text/html;charset=UTF-8 < Content-Length: 80 < Date: Tue, 01 Mar 2022 10:00:53 GMT < #* Connection #0 to host adminui.example.com left intact <html><head><title>Error</title></head><body>Internal Server Error</body></html>
Environment
AdminUI 12.8SP5 on RedHat 7;
Policy Server 12.8SP5 on RedHat 7;
Resolution
Put the code to get the token outside the loop code like this:
#!/bin/bash
myToken=`curl -k -u siteminder:<password> -X POST https://adminui.example.com:8443/ca/api/sso/services/login/v1/token | grep sessionkey | gawk -F: '{print $2}' | grep "\"" | gawk -F\" '{print $2}'`
if [ -z "myToken" ]; then echo "null";else echo "Token :"; echo $myToken; for i in `seq -w 1000`; do curl -k -H "Authorization: Bearer $myToken" -H "Content-Type: application/json; charset=UTF-8" -X POST --data @agent$i.json https://adminui.example.com:8443/ca/api/sso/services/policy/v1/SmAgents -v; done;
Each instantiated session to handle a REST API call requires intensive memory.
As such, a single session is necessary when executing a batch.
This session should be renewed every 15 minutes as stated in the documentation (1).
Additional Information
Feedback
