API Gateway: Log Viewer in Policy Manager showing logs for only one node in cluster and missing all other nodes

Products

• CA API Gateway

Issue/Introduction

This article will discuss the issue where when viewing logs in Policy Manager, it only shows logs from one node and no other nodes in a cluster of multiple nodes. And when trying to view the SSG logs on that one working node, it will often show this warning in the log file:

WARNING 1234 com.l7tech.server.log.SinkManager: Unexpected error during log list/read from remote node '<remote server name>':Invocation of method [public abstract java.util.Collection com.l7tech.gateway.common.log.LogAccessAdmin.findAllFilesForSinkByNode(java.lang.String,com.l7tech.objectmodel.Goid) throws com.l7tech.objectmodel.FindException] failed in HTTP invoker remote service at [https://securespangateway/ssg/cluster/LogAccessAdmin]; nested exception is java.security.AccessControlException: Cluster request disallowed; missing or invalid credentials.

Environment

This article applies to all supported versions of the API Gateway.

Cause

Usually this is due to one of two things:

1. Misconfiguration of Listen Port 2124.
2. Network issue, port 2124 is blocked.

Resolution

To resolve:

1. Ensure all traffic can flow between nodes via port 2124. The port must be exposed in any firewalls or other network devices as needed. Port 2124 must not be blocked.
2. Ensure that Listen Port 2124 on the Gateway (via Policy Manager) is enabled, has Optional set for Client Authentication (SSL/TLS tab), and has the bottom two features enabled for inter-node communications, and finally has TLSv1.1 and TLSv1.2 enabled.