Troubleshooting the Access Gateway Services refuses to start up normally after a clean shutdown
Article ID: 369379
Updated On:
Products
Issue/Introduction
After a clean shutdown, the Access Gateway would not start with error
Access Gateway Server: failed to start address in use
Environment
Applicable to Access Gateway 12.8.x
Applicable to Linux
Cause
The logs show:
[30/May/2024:16:44:48-792] [INFO] - Local apps SSL port not enabled
[30/May/2024:16:44:49-047] [ERROR] - InitCatalina failed ('Protocol handler initialization failed')
[30/May/2024:16:44:49-047] [ERROR] - org.apache.catalina.LifecycleException: Protocol handler initialization failed
[30/May/2024:16:44:49-047] [ERROR] - at org.apache.catalina.connector.Connector.initInternal(Connector.java:1049)
[30/May/2024:16:44:49-047] [ERROR] - at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:136)
[30/May/2024:16:44:49-048] [ERROR] - at org.apache.catalina.core.StandardService.initInternal(StandardService.java:556)
[30/May/2024:16:44:49-048] [ERROR] - at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:136)
[30/May/2024:16:44:49-048] [ERROR] - at org.apache.catalina.core.StandardServer.initInternal(StandardServer.java:1042)
[30/May/2024:16:44:49-048] [ERROR] - at org.apache.catalina.util.LifecycleBase.init(LifecycleBase.java:136)
[30/May/2024:16:44:49-048] [ERROR] - at org.apache.catalina.startup.Tomcat.init(Tomcat.java:475)
[30/May/2024:16:44:49-048] [ERROR] - at com.netegrity.proxy.ProxyServer.initCatalina(Unknown Source)
[30/May/2024:16:44:49-048] [ERROR] - at com.netegrity.proxy.ProxyServer.init(Unknown Source)
[30/May/2024:16:44:49-048] [ERROR] - at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[30/May/2024:16:44:49-048] [ERROR] - at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[30/May/2024:16:44:49-048] [ERROR] - at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[30/May/2024:16:44:49-048] [ERROR] - at java.lang.reflect.Method.invoke(Method.java:498)
[30/May/2024:16:44:49-049] [ERROR] - at com.netegrity.proxy.ProxyBootstrap.execute(Unknown Source)
[30/May/2024:16:44:49-049] [ERROR] - at com.netegrity.proxy.ProxyBootstrap.main(Unknown Source)
[30/May/2024:16:44:49-049] [ERROR] - Caused by: java.net.BindException: Address already in use
[30/May/2024:16:44:49-049] [ERROR] - at sun.nio.ch.Net.bind0(Native Method)
[30/May/2024:16:44:49-049] [ERROR] - at sun.nio.ch.Net.bind(Net.java:461)
[30/May/2024:16:44:49-049] [ERROR] - at sun.nio.ch.Net.bind(Net.java:453)
[30/May/2024:16:44:49-049] [ERROR] - at sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:222)
[30/May/2024:16:44:49-049] [ERROR] - at org.apache.tomcat.util.net.NioEndpoint.initServerSocket(NioEndpoint.java:273)
[30/May/2024:16:44:49-049] [ERROR] - at org.apache.tomcat.util.net.NioEndpoint.bind(NioEndpoint.java:228)
[30/May/2024:16:44:49-050] [ERROR] - at org.apache.tomcat.util.net.AbstractEndpoint.bindWithCleanup(AbstractEndpoint.java:1213)
[30/May/2024:16:44:49-074] [ERROR] - at org.apache.tomcat.util.net.AbstractEndpoint.init(AbstractEndpoint.java:1226)
[30/May/2024:16:44:49-074] [ERROR] - at org.apache.coyote.AbstractProtocol.init(AbstractProtocol.java:603)
[30/May/2024:16:44:49-075] [ERROR] - at org.apache.catalina.connector.Connector.initInternal(Connector.java:1046)
[30/May/2024:16:44:49-097] [ERROR] - ... 14 more
[30/May/2024:16:44:49-097] [ERROR] - ProxyServer initialization failed.
[30/May/2024:16:44:49-097] [ERROR] - Config File: '/apps/CA/secure-proxy/proxy-engine/conf/server.conf')
The java.net.BindException comes from the Tomcat server. And it says the address is already in use, meaning it is trying to bind to the address and port of the server.
Resolution
The default Tomcat ports are
|
Port number for Tomcat HTTP requests
|
Defines the port that listens for the HTTP requests from Tomcat.
Default
: 8080 |
|
Port number for Tomcat SSL requests
|
Defines the port that listens for the SSL requests from Tomcat.
Use a port number above 1024 for installations on UNIX using non-root users.
Default
: 543 |
|
Port number for Tomcat shutdown requests
|
Defines the port that listens for the shutdown requests from Tomcat.
Default
: 8005 |
|
Port number of AJP connector
|
Defines the port number of AJP connector that is used by Apache for forwarding requests to Tomcat.
Default
: 8009 |
In order to check if there something listening on the port preventing Tomcat from binding, use this command:
# netstat -anb | grep 8009
Ex.
# netstat -anp |grep 8009
tcp6 0 0 ::1:8009 :::* LISTEN 4257/java
This is showing a java process with PID 4257. Now check what that PID is like below:
# ps -ef |grep 4257
nobody 4257 1 0 Apr03 ? 11:45:48 /path/CA/jdk1.8.0_251/bin/java -ms256m -mx1024m -server -XX:MaxMetaspaceSize=256M -Dcatalina.base=/path/CA/secure-proxy/Tomcat -Dcatalina.home=/path/CA/secure-proxy/Tomcat -Djava.io.tmpdir=/path/CA/secure-proxy/Tomcat/temp -DHTTPClient.log.mask=0 -DHTTPClient.Modules=HTTPClient.RetryModule|org.tigris.noodle.NoodleCookieModule|HTTPClient.DefaultModule -Dlogger.properties=/path/CA/secure-proxy/Tomcat/properties/logger.properties -Djava.endorsed.dirs=/path/CA/secure-proxy/Tomcat/endorsed -Dfile.encoding=UTF8 -DIWACONFIGHOME=/path/CA/secure-proxy/proxy-engine/conf/sts-config/globalconfig -DNETE_WA_ROOT= -DPWD=/path/CA/secure-proxy -classpath /path/CA/secure-proxy/Tomcat/bin/proxybootstrap.jar:/path/CA/secure-proxy/Tomcat/properties:/path/CA/secure-proxy/resources:/path/CA/jdk1.8.0_251/lib/tools.jar:/path/CA/jdk1.8.0_251/lib/tools.jar:/path/CA/secure-proxy/Tomcat/bin/bootstrap.jar:/path/CA/secure-proxy/Tomcat/lib/smi18n.jar:/path/CA/secure-proxy/agentframework/java/bc-fips-1.0.2.3.jar com.netegrity.proxy.ProxyBootstrap -config /path/CA/secure-proxy/proxy-engine/conf/server.conf
root 28435 28349 0 21:31 pts/0 00:00:00 grep --color=auto 4257
Simply change 8009 to any other ports you want to check
In this particular case, the process was still running after shutdown and it was necessary to manually kill the process before restarting the Access Gateway
Feedback
