Gateway service not starting after upgrade from 10.0 to 10.1 Software Form Factor
search cancel

Gateway service not starting after upgrade from 10.0 to 10.1 Software Form Factor

book

Article ID: 241205

calendar_today

Updated On:

Products

CA API Gateway

Issue/Introduction

We tried to upgrade our software gateway from 10 to 10.1 server and it is not coming up after the upgrade. 

And also below are the error in the controller logs:

2022-05-03T00:28:32.006+0800 INFO    11 com.l7tech.server.processcontroller.ProcessController: default stopping
2022-05-03T00:30:51.894+0800 CONFIG  1 com.l7tech.server.processcontroller: Logging initialized with defaults from 'jar:file:/opt/SecureSpan/Controller/Controller.jar!/com/l7tech/server/processcontroller/resources/logging.properties'
2022-05-03T00:30:51.987+0800 INFO    1 com.l7tech.server.processcontroller.BootstrapConfig: Starting configuration bootstrap Layer7 API Gateway 10.1.00 build 11620, built 20210706173756 by teamcity at ${env.COMPUTERNAME}
2022-05-03T00:30:51.993+0800 INFO    1 com.l7tech.server.processcontroller.BootstrapConfig: Configuration bootstrap complete.
2022-05-03T00:30:52.477+0800 CONFIG  1 com.l7tech.server.processcontroller: Logging initialized with defaults from 'jar:file:/opt/SecureSpan/Controller/Controller.jar!/com/l7tech/server/processcontroller/resources/logging.properties'
2022-05-03T00:30:52.548+0800 INFO    1 com.l7tech.server.processcontroller.ProcessControllerDaemon: Starting Process Controller Layer7 API Gateway 10.1.00 build 11620, built 20210706173756 by teamcity at ${env.COMPUTERNAME}
2022-05-03T00:30:53.160+0800 INFO    1 com.l7tech.server.processcontroller.ConfigServiceImpl: Using java binary: /opt/SecureSpan/JDK/bin/java
2022-05-03T00:30:54.008+0800 INFO    1 com.l7tech.server.processcontroller.ConfigServiceImpl: Remote node management disabled
2022-05-03T00:30:54.009+0800 INFO    1 com.l7tech.server.processcontroller.ConfigServiceImpl: No user trusted certificates for patch verification.
2022-05-03T00:30:54.061+0800 INFO    1 com.l7tech.server.processcontroller.ConfigServiceImpl: Detected node 'default'.
2022-05-03T00:30:54.101+0800 INFO    1 com.l7tech.server.processcontroller.ConfigServiceImpl: No monitoring configuration found in /opt/SecureSpan/Controller/var/currentMonitoringConfig.xml
2022-05-03T00:30:54.203+0800 INFO    1 com.l7tech.server.processcontroller.PCServletContainer: Added default IPv4 SSL connector.
2022-05-03T00:30:55.559+0800 INFO    1 com.l7tech.server.processcontroller.ProcessController: Getting API port from /opt/SecureSpan/Gateway/node/default/var/processControllerPort
2022-05-03T00:30:55.775+0800 INFO    1 com.l7tech.server.processcontroller.p: default starting
2022-05-03T00:31:00.809+0800 INFO    1 com.l7tech.server.processcontroller.ProcessController: Getting API port from /opt/SecureSpan/Gateway/node/default/var/processControllerPort
2022-05-03T00:31:05.825+0800 INFO    1 com.l7tech.server.processcontroller.ProcessController: Getting API port from /opt/SecureSpan/Gateway/node/default/var/processControllerPort
2022-05-03T00:31:10.837+0800 INFO    1 com.l7tech.server.processcontroller.ProcessController: Getting API port from /opt/SecureSpan/Gateway/node/default/var/processControllerPort
2022-05-03T00:31:15.847+0800 WARNING 1 com.l7tech.server.processcontroller.p: default exited with status 77
2022-05-03T00:31:15.909+0800 WARNING 1 com.l7tech.server.processcontroller.ProcessController: default crashed on startup with exit code 77
2022-05-03T00:31:15.910+0800 WARNING 1 com.l7tech.server.processcontroller.ProcessController: default crashed on startup; copying its output:

 

Environment

Release : 10.1

Component :

Cause

appdynamics entries added on ssgruntimedefs.sh

# LAYER 7 TECHNOLOGIES
# Defines JAVA_HOME, etc

default_java_opts="-server "

if [ ! -f "/etc/sysconfig/network" -o  "$(grep "^NETWORKING_IPV6=yes" /etc/sysconfig/network 2>/dev/null)" ]
then
  default_java_opts="$default_java_opts -Djava.net.preferIPv4Stack=false "
else
  default_java_opts="$default_java_opts -Djava.net.preferIPv4Stack=true "
fi

default_java_opts="$default_java_opts -Djava.security.policy=${SSG_HOME}/runtime/etc/ssg.policy"
default_java_opts="$default_java_opts -Djava.security.egd=file:/dev/./urandom"
default_java_opts="$default_java_opts -Dfile.encoding=UTF-8 "
default_java_opts="$default_java_opts -Djava.awt.headless=true -XX:CompileThreshold=1500 "
default_java_opts="$default_java_opts -Dcom.l7tech.server.defaultClusterHostname=$(hostname)"
default_java_opts="$default_java_opts -Dcom.sun.management.jmxremote.port=5002"
default_java_opts="$default_java_opts -Dcom.sun.management.jmxremote.rmi.port=5002"
default_java_opts="$default_java_opts -Dcom.sun.management.jmxremote.authenticate=false"
default_java_opts="$default_java_opts -Dcom.sun.management.jmxremote.ssl=false"
# sun.net.inetaddr.ttl and java.util.Arrays.useLegacyMergeSort are system properties that are read during
# JVM start up and need to be specified as command line args to make sure the values are used.
# Do not move sun.net.inetaddr.ttl and java.util.Arrays.useLegacyMergeSort into the system.properties file.
default_java_opts="$default_java_opts -Dsun.net.inetaddr.ttl=30 "
default_java_opts="$default_java_opts -Djava.util.Arrays.useLegacyMergeSort=true"
default_java_opts="$default_java_opts -javaagent:/opt/appdynamics/appagent/javaagent.jar"
default_java_opts="$default_java_opts -Dappdynamics.http.proxyHost=web.prod.proxy.domain.com"
default_java_opts="$default_java_opts -Dappdynamics.http.proxyPort=4200"
default_java_opts="$default_java_opts -Dappdynamics.http.proxyUser=ps123449"
default_java_opts="$default_java_opts -Dappdynamics.http.proxyPasswordFile=/opt/appdynamics/appagent/conf/pswd"
SSG_JAVA_OPTS="$SSG_JAVA_OPTS $default_java_opts";

unset default_java_opts

export SSG_JAVA_OPTS

if ! echo $LD_LIBRARY_PATH | /bin/egrep -s "(^|:)$SSG_HOME/runtime/lib($|:)" >/dev/null ; then
 LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$SSG_HOME/runtime/lib
fi

export LD_LIBRARY_PATH

Resolution

1.please, ensure you follow this steps when doing the upgrade using patches

Patch installation Order -  Proper Patch Install Order
Proper patching for Gateway Upgrade from 10.0 - 10.1 CR01
https://knowledge.broadcom.com/external/article?articleId=234729

2. modified the ssgruntimedefs.sh and leave as follow :

# LAYER 7 TECHNOLOGIES
# Defines JAVA_HOME, etc

default_java_opts="-server "

if [ ! -f "/etc/sysconfig/network" -o  "$(grep "^NETWORKING_IPV6=yes" /etc/sysconfig/network 2>/dev/null)" ]
then
  default_java_opts="$default_java_opts -Djava.net.preferIPv4Stack=false "
else
  default_java_opts="$default_java_opts -Djava.net.preferIPv4Stack=true "
fi

default_java_opts="$default_java_opts -Djava.security.policy=${SSG_HOME}/runtime/etc/ssg.policy"
default_java_opts="$default_java_opts -Djava.security.egd=file:/dev/./urandom"
default_java_opts="$default_java_opts -Dfile.encoding=UTF-8 "
default_java_opts="$default_java_opts -Djava.awt.headless=true -XX:CompileThreshold=1500 "
default_java_opts="$default_java_opts -Dcom.l7tech.server.defaultClusterHostname=$(hostname)"
# sun.net.inetaddr.ttl and java.util.Arrays.useLegacyMergeSort are system properties that are read during
# JVM start up and need to be specified as command line args to make sure the values are used.
# Do not move sun.net.inetaddr.ttl and java.util.Arrays.useLegacyMergeSort into the system.properties file.
default_java_opts="$default_java_opts -Dsun.net.inetaddr.ttl=30 "
default_java_opts="$default_java_opts -Djava.util.Arrays.useLegacyMergeSort=true"

SSG_JAVA_OPTS="$SSG_JAVA_OPTS $default_java_opts";

unset default_java_opts

export SSG_JAVA_OPTS

if ! echo $LD_LIBRARY_PATH | /bin/egrep -s "(^|:)$SSG_HOME/runtime/lib($|:)" >/dev/null ; then
        LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$SSG_HOME/runtime/lib
fi

export LD_LIBRARY_PATH

 

**If gateway not starting properly after change above , do the following :

1. gateway.sh stop

# find -name ssgruntimedefs.sh

2. once identified the folder , replace your current ssgruntimedefs.sh with the file (ssgruntimedefs.sh from a working v10.1  env)  :
please, copy this file to /home/ folder and change the permissions to the file as follow :

chown layer7:layer7 ssgruntimedefs.sh

2. once replaced the ssgruntimedefs.sh on the proper folder, please execute 

# su - gateway

# ./opt/SecureSpan/Gateway/runtime/bin/gateway.sh run

3. we will check if gateway is running or writing something to ssg 

ps -fea |grep Gateway.jar

3. you can try to start the gateway in console mode, showing all details on the startup process to check if there is some error. 

example :

./gateway.sh start -console

 

Powered by Wolken Software