Rally - On-premises: The Reporting Service Is Currently Unavailable
search cancel

Rally - On-premises: The Reporting Service Is Currently Unavailable

book

Article ID: 94782

calendar_today

Updated On:

Products

Rally On-Premise

Issue/Introduction

The Agile Central on-premise appliance is reporting the following error:

The Reporting Service Is Currently Unavailable
Please check back shortly. 

Environment

Release: 2018.1 and below
Component: ACPREM

Cause

This error can be caused by a custom report that causes the analytics service to run out of memory.  Some poorly written queries can cause the analytics service to use it's memory and not be able to correctly clean up after itself.

This can be verified by gathering a log bundle from the Control Panel and looking at the analytics-startup.log file.
Searching for "Caused by" will typically locate the first instance of the problem and the message may appear as follows:

Caused by: java.lang.OutOfMemoryError: Java heap space

Resolution

Immediate remediation is to restart the Agile Central services.

The best course of action is to locate the offending custom report and ensure the query is not causing performance issues.

However, if this report can not be rewritten, then it may be necessary to increase the memory assigned to the analytics service.  You are welcome to open a support case to perform this action if you are not comfortable working on the command line or if you do not have root access to your appliance.

Increasing analytics memory allocation

  1. Log in as root to your appliance
  2. Edit /home/appuser/etc/rally_node_overrides.cfg using your preferred text editor
    • Example: # nano /home/appuser/etc/rally_node_overrides.cfg
  3. Append the following line: 
    • set(:analytics_max_memory) { "1024m" }
  4. Restart the Agile Central services
  5. Verify that the setting has been updated using 'ps'
    1. # ps ax | grep analytics | grep 1024m
    2. 20475 ?        Sl    92:34 /usr/lib/jvm/jdk1.7.0_80/bin/java -Dkill-tag=false -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=5101 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false -DXSSGUARD_ENABLED=true -Danalytics.jdbc.username=rallyapp_analytics -Danalytics.jdbc.password=<PASSWORD> -Danalytics.jdbc.service.name=@//localhost:1521/f4wp1 -Danalytics.bootstrap.db=true -DANALYTICS_BIND_ADDRESS=rallyonprem-oraclese -DANALYTICS_SERVER_PORT=8080 -DLOG_DIR=/home/appuser/domains/analytics/logs -Danalytics.log.dir=/home/appuser/domains/analytics/logs -Dlogback.configurationFile=file:///home/appuser/appserver-config-latest/analytics/etc/logback.xml -Djetty.home=/home/appuser/appserver-config-latest/jetty-9 -Djetty.base=/home/appuser/appserver-config-latest/jetty-9 -Drally.jetty.home=/home/appuser/appserver-config-latest/analytics -Djetty.class.path=/home/appuser/appserver-config-latest/migration/ojdbc16.jar -Djava.security.egd=file:///dev/urandom -Djava.io.tmpdir=/home/appuser/appserver-config-latest/analytics/tmp -DSTACK=onprem -Xms1024m -Xmx1024m -Xloggc:/home/appuser/domains/analytics/logs/gc.log -XX:MaxPermSize=256m -XX:NewRatio=3 -XX:+UseConcMarkSweepGC -XX:+UseCompressedOops -XX:+PrintGCApplicationStoppedTime -XX:+PrintGCApplicationConcurrentTime -XX:+PrintGCDetails -XX:+PrintGCTimeStamps -XX:+PrintGCDateStamps -XX:-TraceClassUnloading -jar /home/appuser/appserver-config-latest/jetty-9/start.jar /home/appuser/appserver-config-latest/analytics/etc/jetty.xml
       


In general, it is not recommended to increase beyond 1024m as the time spent during memory cleanup can begin to negatively impact the performance of the service, however there may be times where this is unavoidable.

If you continue to experience errors, then the next course of action would be to open a support ticket.

 

Additional Information

Keywords: rally agile central

Powered by Wolken Software