AAI: Log and Configuration Gathering Checklist for Support Cases
search cancel

AAI: Log and Configuration Gathering Checklist for Support Cases

book

Article ID: 455772

calendar_today

Updated On:

Products

Automation Analytics & Intelligence

Issue/Introduction

 This article provides a comprehensive checklist for customers and support engineers to gather necessary log and configuration files for AAI (Automic Automation Intelligence) support cases.
It covers both VM/traditional-installer and Kubernetes/OpenShift deployments for AAI versions 24.x and 26.0.0+.

Environment

Automic Automation Intelligence (AAI) 
Versions: AAI 24.x, AAI 26.0.0+ 
Environment: VM/traditional-installer, Kubernetes/OpenShift (Helm)

Cause

N/A - This is a checklist for proactive data collection to aid in problem resolution.

Resolution

Prerequisites

  • Shell access to your AAI installation (VM), or kubectl/oc access to the correct namespace (Kubernetes/OpenShift) — confirm with kubectl config current-context (or oc whoami).
  • (Kubernetes/OpenShift only) helm, configured against the correct cluster, plus your Helm release name/namespace — confirm with helm version, then see item 1 below if you don't already know the release name/namespace.
  • curl (built into Windows 10/11, macOS, and most Linux distros by default) for the version-check commands in item 1, or just a browser.
  • Beyond that, no specialized tools are required — this topic otherwise only uses directory listing, tar/zip, and text search that ship with the OS.

Which environment and version are you on?

Confirm both before using this checklist:

  • VM/traditional installer or Kubernetes/OpenShift (Helm) — file locations and how configuration actually takes effect differ genuinely between the two (see item 3).
  • 24.x or 26.0.0+ — a small number of items below (marked explicitly) only exist starting in 26.0.0.

1. Checking the version

Unlike some Broadcom products, AAI does not have a standalone command-line "-version" flag on its launcher scripts. Version information is exposed through the product itself instead:

Easiest — no command needed: the AAI Web UI shows the running product version on the Login page and in the top app header/bar once logged in.

Scriptable check (REST):

  • GET /aai/api/properties/jawsVersion — the product version.
  • GET /aai/api/properties/aaiBuildId — the build ID.

On 26.0.0+, these two endpoints are deliberately exposed without authentication (an explicit, documented allowlist in the product's own web configuration — not an oversight), so a plain curl works directly:

curl -sk https://<host>:<port>/aai/api/properties/jawsVersion
curl -sk https://<host>:<port>/aai/api/properties/aaiBuildId

On 24.x, the same two endpoints exist but require an authenticated session — there is no unauthenticated allowlist on 24.x, so a bare curl above will get rejected.
On 24.x, use the Web UI (Login page / app header) instead, or supply your session cookie/credentials to curl the same way you would for any other authenticated AAI REST call.

A fuller system-details view (server ID, product version, database host/port/schema version, and basic health) is available at:

GET /aai/api/systems/data

This one requires an authenticated session on both 24.x and 26.0.0+.

If the Web UI and REST API are both unreachable: on startup, AAI writes a line beginning JAWS Version: (followed by the version and build ID) to jaws.log.
This applies to both 24.x and 26.0.0+. Search for the last occurrence in the file, since it's written once per server start and older ones may still be present after a restart:

  • Linux: grep "JAWS Version:" jaws.log | tail -1
  • Windows (PowerShell): Select-String "JAWS Version:" jaws.log | Select-Object -Last 1

See item 3 below for where jaws.log lives in your environment (VM vs. Kubernetes/OpenShift).

If you need the Helm release name/namespace first (Kubernetes/OpenShift): every AAI component's pod name starts with aai-, so you can find the namespace, then the release, before running any of the commands elsewhere in this topic:

kubectl get pods -A | grep aai-
helm list -n <namespace>

(oc get pods --all-namespaces | grep aai- on OpenShift.)

2. Diagnostic/system-details commands

AAI does not have a single all-in-one diagnostic-dump utility. The closest equivalents:

Both VM and Kubernetes/OpenShift:

What you wantHow to get it
Server ID, version, DB connection details, schema version, basic healthGET /aai/api/systems/data (authenticated)

Kubernetes/OpenShift only:

What you wantHow to get it
Pod-level liveness/readiness/health/ready and /health/live on the Config Tool's health-check endpoint (26.0.0+ only — this did not exist on 24.x)
A full picture of what's actually configuredhelm get values <release-name> -n <namespace> -a (see Where Did My Logs and Configuration Go? for why a file inside a running pod is not a reliable source instead)

There is no equivalent of a one-shot "collect everything" script — the file list in item 3 below is what you gather manually (or via your own tar/zip of the directories named).

3. Logs and configuration files to collect, by component

VM/traditional installer and Kubernetes/OpenShift genuinely diverge here — different directories, and on Kubernetes/OpenShift, whether a given configuration file is worth collecting directly from a pod versus from Helm values depends on the specific file.
Use the VM subsection or the Kubernetes/OpenShift subsection below, not both, based on what you confirmed above.

VM / traditional installer

Logs live under $AAI_HOME/log/. Configuration is split across three directories — note how similar the first two names are; they are not the same directory:

  • $AAI_HOME/configuration/ — the 26.0.0+ server configuration directory (new in 26.0.0)
  • $AAI_HOME/jboss/standalone/configuration/ — the legacy configuration directory, still active on both 24.x and 26.0.0+
  • $AAI_HOME/config/ — the Config Tool/Webswing runtime directory, present on both 24.x and 26.0.0+ (see the Webswing subsection below)

On 24.x, everything under "AAI Server" below lives in the legacy directory only — the split configuration/ directory does not exist yet. $AAI_HOME/config/ (Webswing) exists on both versions.

AAI Server

  • Logs: server.logjaws.logalertService.logsimulationService.logreportingService.logcollectorService.logsubscriptions.log, the audit log, predictions.log (the last two may legitimately not exist yet on a healthy system — see Where Did My Logs and Configuration Go?)
  • Config (26.0.0+ new directory): configuration/aai.propertiesconfiguration/aai-jboss-log4j2.ymlconfiguration/opensearch.properties
  • Config (legacy directory, both 24.x and 26.0.0+): jboss/standalone/configuration/standalone.xmljaws-database.xmljaws.propertiesehcache.xmlapplication.keystoreaai-server.p12root-ca.p12
  • JVM heap/GC flags (install root, not either configuration directory): aai.sh.vmoptions (Linux) or aai.vmoptions (Windows)

eEM / EIAM (identity/authorization)

  • Logs: aai.eem.log (AAI's own eEM-related logging), plus the eEM SDK's own eem.sdk.logeem.network.logeem.perf.log — request all four together, especially on a "login works, no data" case
  • Config: configuration/eiam.config (26.0.0+ new directory) or the equivalent legacy-directory location on 24.x

Frontend Gateway (26.0.0+ only — this component did not exist on 24.x)

  • Log: frontendGateway.log
  • Config: configuration/frontend-gateway-log4j2.yml

Denormalizer (26.0.0+ only)

  • Log: denormalizer.log
  • Config: configuration/denormalizer-log4j2.yml

MCP Service (26.0.0+ only)

  • Log: mcp.log
  • Config: configuration/mcpserver-log4j2.yml

AI Insights (26.0.0+ only)

  • Log: aiinsights.log
  • Config: configuration/ai-insights-log4j2.yml

Automation AI (26.0.0+ only)

  • Log: automationai.log
  • Config: configuration/automation-ai-log4j2.yml, plus configuration/AutomationAI/application.properties (AI provider settings)

Webswing / Config Tool web UI (both 24.x and 26.0.0+)

Webswing is the framework AAI bundles to serve the desktop Config Tool GUI inside a browser. It runs as its own managed service (start/stop/status via aai.sh/aai.bat, alongside the other services above), with its own runtime directory, log, and TLS configuration — all under $AAI_HOME/config/, not $AAI_HOME/configuration/.

  • Logs:
    • Linux/macOS: config/webswing.out (rotated copies land under config/logs/outarchive/ once the current file passes 100 MB)
    • Windows: config/webswing_launcher.log (start/stop/status actions) and config/logs/webswing.log (the running server's own log)
  • Config: config/webswing.properties (connection secret used to sign session tokens), config/jetty.properties (listener host/ports and TLS keystore/truststore paths — see item 5), config/webswing.config (app definition)
  • Default ports: 8088 (HTTP) / 8488 (HTTPS) — see item 4

IWS z Connector (both 24.x and 26.0.0+)

The IWS z Connector links AAI to IBM Workload Scheduler for z/OS. It runs as its own separate installed service, with its own install directory and its own Tomcat/Catalina container — separate from the main AAI Server install.

  • Logs: located in <Install Dir>/logs, named <SchedulerID>_<Type>_iwscon.log and <SchedulerID>_<Type>_iwscon-httpclient.log
    • <SchedulerID> identifies the specific scheduler connection — look it up in the AAI database's JOBSCHEDULER table
    • <Type> is one of: EV (Events), DL (Definitions and Calendars), HB (Heart Beat — periodic connectivity check back to AAI)
    • The plain _iwscon.log file shows processing/progress for that type; the matching _iwscon-httpclient.log file shows the JSON payloads exchanged with AAI for that type
    • Example, for scheduler ID AXT5hhRGB7k=:
      • AXT5hhRGB7k=_EV_iwscon.log — event processing progress; AXT5hhRGB7k=_EV_iwscon-httpclient.log — JSON event data sent back to AAI
      • AXT5hhRGB7k=_DL_iwscon.log — progress of jobs/calendars being added to AAI; AXT5hhRGB7k=_DL_iwscon-httpclient.log — JSON sent to AAI for definitions/calendars
      • AXT5hhRGB7k=_HB_iwscon.log / AXT5hhRGB7k=_HB_iwscon-httpclient.log — heart-beat connectivity-check logs
  • localhost_access_log.<date>.txt (same directory) — incoming API calls from AAI to the Connector and their HTTP status
  • catalina.out / catalina*.log (same directory) — service startup and status

OpenSearch (search/indexing backend — third-party, bundled connection details only)

  • AAI does not manage OpenSearch's own lifecycle or logs on a VM install — only connection details are stored, in configuration/opensearch.properties
  • OpenSearch's own logs and sizing are otherwise outside AAI's configuration surface — see Where Did My Logs and Configuration Go? for what AAI itself tracks about it

Kubernetes / OpenShift

Logs are per pod on shared storage (<per-component-volume>/<component>/log/<pod-name>/). Configuration is a mix
— some of it is genuinely persistent on shared storage and worth collecting directly from a pod, some of it is rebuilt fresh from environment variables/secrets on every pod start and is better collected as 
Helm values instead (helm get values <release-name> -n <namespace> -a). Which applies is called out per component below — don't assume one rule for the whole environment.

AAI Server

  • Logs: server.logjaws.logalertService.logsimulationService.logreportingService.logcollectorService.logsubscriptions.log, the audit log, predictions.log (the last two may legitimately not exist yet on a healthy system
    — see Where Did My Logs and Configuration Go?)
  • Config — persistent, collect directly from the pod: the whole legacy configuration directory lives on shared storage and survives pod restarts, so 
    standalone.xmljaws-database.xmljaws.propertiesehcache.xmlaai-jboss-log4j2.yml, and the internal mTLS keystores (item 5) are all genuinely present and worth pulling with kubectl cp/kubectl exec 
    — this is the same underlying directory the Config Tool/Webswing pod also mounts (see below)
  • Config — not persistent, collect Helm values instead: aai.propertiesopensearch.properties, and eiam.config are seeded from the container image at pod start and explicitly cleaned back out of shared storage afterward
    — pulling them from the pod just gets you the image defaults, not anything customer-specific

eEM / EIAM (identity/authorization)

  • Logs: aai.eem.log (AAI's own eEM-related logging), plus the eEM SDK's own eem.sdk.logeem.network.logeem.perf.log — request all four together, especially on a "login works, no data" case
  • Config: collect Helm values (eiam.config is not persistent inside the pod)

Frontend Gateway (26.0.0+ only — this component did not exist on 24.x)

  • Log: frontendGateway.log — first confirm the AAI_LOG_DIR environment variable is actually set in the running pod before assuming this is retrievable from shared storage; if not,
    use kubectl logs <gateway-pod> -n <namespace> instead (see Where Did My Logs and Configuration Go?)
  • Config: collect Helm values

Denormalizer (26.0.0+ only)

  • Log: denormalizer.log
  • Config: collect Helm values

MCP Service (26.0.0+ only)

  • Log: mcp.log
  • Config: collect Helm values

AI Insights (26.0.0+ only)

  • Log: ai-insights.log — note the different spelling from the VM build; the wrong spelling returns "no such file"
  • Config: collect Helm values

Automation AI (26.0.0+ only)

  • Log: automationAI.log — note the different spelling from the VM build
  • Config: collect Helm values (AI provider settings normally found in AutomationAI/application.properties on a VM install)

Webswing / Config Tool web UI (both 24.x and 26.0.0+)

Webswing/Config Tool is deployed as its own pod, Deployment, and Service in Kubernetes/OpenShift — component/pod name aai-configtool.

  • Log: per-pod shared storage, aai-configtool/log/<pod-name>/ (mirrors the <per-component-volume>/<component>/log/<pod-name>/ pattern above)
  • Config — persistent, collect directly from the pod: this pod mounts the same shared configuration directory as the AAI Server pod above,
    so jetty.properties and webswing.config (item 5) are genuinely present on shared storage, not reset from the image
  • Secrets: the Webswing admin password is stored in the aai-configtool-secret Secret (key webswing-password);
    Webswing's own inbound TLS keystore/truststore material is in the aai-webswing-service-tls Secret
    — collect Helm values to confirm the exact secret names in use, since both are configurable
  • Ports: same defaults as VM (8088 HTTP / 8488 HTTPS, see item 4), exposed via the aai-configtool Service

IWS z Connector

Deployed as its own pod, Deployment, and Service — component/pod name aai-iws-connector.

  • Logs: persistent, per-pod shared storage
    — aai-iws-connector/log/<pod-name>/
     (mirrors the <per-component-volume>/<component>/log/<pod-name>/ pattern above),
    mounted into the pod at /opt/aai-iws-connector/log.
    Same file-naming convention as the VM build (item 3A): 
    <SchedulerID>_<Type>_iwscon.log<SchedulerID>_<Type>_iwscon-httpclient.loglocalhost_access_log.<date>.txtcatalina.out/catalina*.log 
    — these genuinely survive pod restarts, unlike some of the other components' configuration above.
  • Config — persistent, collect directly from the pod: mounted at /opt/aai-iws-connector/config on the same shared storage; not reset from the image.
  • Optional SFTP-based file exchange with the scheduler, if enabled for this install, uses a separate persistent volume mounted at /opt/aai-iws-connector/mdp/jobs and /opt/aai-iws-connector/mdp/events 
    — collect Helm values to confirm whether this is enabled for the customer's release.
  • Ports: 9876 (HTTP), plus an optional inbound mTLS listener on 9877 when internal mTLS is enabled (see item 5) — see item 4.

OpenSearch (search/indexing backend — third-party)

  • Retrieve OpenSearch's own log with kubectl logs <opensearch-pod> -n <namespace>
  • OpenSearch's own logs and sizing are otherwise outside AAI's configuration surface — see Where Did My Logs and Configuration Go? for what AAI itself tracks about it

4. Ports

Useful for checking connectivity, firewall rules, and reverse-proxy/load-balancer configuration. Applicability differs by version.

ComponentDefault port(s)Notes
Browser-facing HTTPS (24.x only)8443Terminated directly by the AAI server itself, via the legacy configuration directory's standalone.xml (https-listener/socket-binding). Does not work this way on 26.0.0+.
Frontend Gateway — browser-facing (26.0.0+ only)8080 (HTTP) / typically 8443 once TLS is enabledOn 26.0.0+, browser-facing HTTPS is terminated here, not by the AAI server — see item 5. Configured via SSL_* and port properties in aai.properties (new configuration directory).
AAI server internal listener (26.0.0+ only)8081Internal-only; not browser-facing on 26.0.0+.
AAI server internal mTLS listener (26.0.0+ only)4443Internal service-to-service TLS between AAI's own components — see item 5.
Cache/data processor (26.0.0+ only)8181 
Denormalizer (26.0.0+ only)8183 
MCP Service (26.0.0+ only)8184 
Automation AI (26.0.0+ only)8185 / 8186Nominal service port 8185; the running launcher actually listens on 8186.
AI Insights (26.0.0+ only)8187 
Webswing / Config Tool web UI (both 24.x and 26.0.0+)8088 (HTTP) / 8488 (HTTPS)VM: set in config/jetty.properties. Kubernetes/OpenShift: same defaults, exposed via the aai-configtool Service.
IWS z Connector (Kubernetes/OpenShift)9876 (HTTP) / 9877 (optional mTLS, when enabled)Exposed via the aai-iws-connector Service. VM port not confirmed — check the connector's own startup log/config on that install.

Kubernetes/OpenShift: these aren't bound directly on a host — each is exposed per-Service through the Helm chart. Collect helm get values <release-name> -n <namespace> -a 
and cross-reference against the table above rather than expecting a host-level port scan to show them directly.

5. Keystores / TLS configuration

Where certificates and keys live differs by version and by which connection you're looking at: browser-facing HTTPS, internal service-to-service TLS, or Webswing's own listener.

Browser-facing HTTPS:

  • 24.x: the AAI server terminates HTTPS itself. Keystore: application.keystore, in the legacy configuration directory, referenced from standalone.xml.
  • 26.0.0+: the Frontend Gateway terminates HTTPS instead (see item 4). Its keystore/truststore path, type, and password are set via SSL_KEYSTORE_PATHSSL_KEYSTORE_PASSWORDSSL_KEYSTORE_TYPESSL_TRUSTSTORE_PATHSSL_TRUSTSTORE_PASSWORDSSL_TRUSTSTORE_TYPE, and related SSL_* properties in aai.properties (new configuration directory).

Internal service-to-service TLS (mTLS, 26.0.0+ only): AAI's own components (AAI server, Frontend Gateway, Denormalizer, MCP Service, AI Insights, Automation AI, Webswing, and others) authenticate to each other using certificates produced together by the same shipped certificate-generation utility, written into the legacy configuration directory:

  • root-ca.p12 — the shared trust store every internal service trusts (self-signed by default, or your own CA if one was supplied during setup)
  • one <service>.p12 per component — aai-server.p12frontend-gateway.p12denormalizer-service.p12mcp-service.p12ai-insights-service.p12automation-ai-service.p12webswing-service.p12, and others — each holding that component's private key and certificate

Webswing's own listener: independent of the above. Its HTTP/HTTPS ports, keystore, and truststore are set in config/jetty.properties, pointing by default at config/ssl/keystore.p12 and config/ssl/truststore.p12. Its session-signing connection secret lives separately, in config/webswing.properties.

Encrypted passwords are expected, not a defect. Several values above — notably the Frontend Gateway's SSL_KEYSTORE_PASSWORD/SSL_TRUSTSTORE_PASSWORD in aai.properties — are stored encrypted, decrypted at runtime using a local key file. Finding an encrypted-looking string when you open the file directly is normal.

Kubernetes/OpenShift: this splits the same way configuration does in item 3 — don't assume everything is ephemeral:

  • Persistent, collect directly from the pod: the internal mTLS keystores (root-ca.p12 and the per-service <service>.p12 files, including webswing-service.p12) live on the same shared configuration storage as the AAI Server pod's legacy configuration directory, and survive pod restarts.
  • Not persistent, collect Helm values/secrets instead: the Frontend Gateway's browser-facing keystore/truststore (SSL_KEYSTORE_*/SSL_TRUSTSTORE_*) and
    Webswing's admin password are supplied via Kubernetes Secrets referenced from the Helm chart (for Webswing specifically, the aai-configtool-secret Secret's webswing-password key
    and the aai-webswing-service-tls Secret) rather than files sitting on shared storage. Collect helm get values <release-name> -n <namespace> -a and check how the TLS-related values are set there.

6. Priority list: main configuration files to capture during initial investigation

If you can only grab a handful of files before opening a case, or want to sanity-check that nothing critical is missing before closing out an initial round of collection, prioritize these first (all VM/traditional-installer paths — see the Kubernetes/OpenShift note below if that's your environment):

Both 24.x and 26.0.0+:

  • standalone.xml and jaws-database.xml (legacy configuration directory)
  • aai-jboss-log4j2.yml (26.0.0+) or log4j2.yml (24.x) — logging configuration, needed to interpret any gaps in the logs you collect
  • config/webswing.properties and config/jetty.properties — Config Tool/Webswing

26.0.0+ additionally:

  • aai.properties (new configuration directory) — the single most information-dense file: ports, TLS settings, and general server configuration all in one place
  • opensearch.properties
  • eiam.config

Keystores/certificates (name the specific file(s) you're attaching, per item 5):

  • application.keystore (24.x), or the files pointed to by SSL_KEYSTORE_PATH/SSL_TRUSTSTORE_PATH in aai.properties (26.0.0+)
  • root-ca.p12 plus whichever specific <service>.p12 file is relevant to the issue (26.0.0+)

Kubernetes/OpenShift: capture helm get values <release-name> -n <namespace> -a, plus these files directly from the AAI Server pod's shared configuration storage
— they're genuinely persistent, not reset from the image (see item 3): 
standalone.xmljaws-database.xmljaws.propertiesehcache.xmlaai-jboss-log4j2.ymlroot-ca.p12, and the specific <service>.p12 relevant to the issue.

7. Misc. tips

AAI does not use a standardized error-code prefix convention. Unlike some Broadcom products that prefix messages with a fixed code (e.g. CAUAJM_E_...),
AAI's log lines are plain text
— timestamp, level, logger name, message
— with no coded scheme to search for. When searching a bundle, search for the log level (ERRORWARN) and the relevant component/logger name instead of a message-code pattern.

A missing log file is often expected, not a defect. Several files (the audit log, the prediction-engine log) are only created the first time something is actually written to them — see Where Did My Logs and Configuration Go?, before treating an absent file as evidence of a fault.

Confirm install type and environment before naming a file in a request — several file names differ between VM/standalone-installer and Kubernetes/OpenShift builds (item 3), and 24.x deployments simply don't have several of the 26.0.0+ services at all.

Additional Information