Excessive pod / container logging management

Products

• CA API Gateway

Issue/Introduction

The Layer7 API Gateway container logs a huge amount of data to the pod’s console, with the result that all this logging is also stored within OpenShift/Kubernetes and processed by Kibana. The volume of logging is so great that the API Gateway alone uses up almost all the log space available on the shared cluster. We would like to reduce this volume of logging.

It appears that the audit logging in particular is responsible for the large volume of logging, given the many log lines beginning with {‘package’:"com.l7tech.external.assertions.comparison.server. ServerComparisonAssertion‘,“level”:’INFO" 

Environment

API Gateway 11.1

Resolution

System Audits

These are internal messages that are constantly generated in the background by the Gateway. These messages typically describe "housekeeping" tasks such as: server starting , license updated, connecting to a JMS endpoint, etc.

System audit events are normally rated 

Fine, Finer,

 or 

Finest

 in the severity scale, although some may be rated 

Info

. System audit events are always available in the audit event log (you will need to set the filter slider to "All" to see all these events).

You have no control over system audits: they happen automatically, without requiring any assertions.

 

Administrative Audits

These are messages that occur when an administrative action is performed via the Policy Manager , via an administrative API, or through the Enterprise Service Manager. Examples of such actions include: publishing or updating a policy, creating a user, etc.

You can control which of these messages you want to save by using the 

audit.adminThreshold

 cluster property. By default, all messages at level 

Info

 or higher are logged.

 

Policy Message Audits

These are messages generated during the processing of a policy. The bulk of these are simply informational messages that have the severity level 

Info

. The more important messages are rated 

Warning

 or 

Severe

. By default, the Gateway is set to save only 

Warning

 or 

Severe

 messages; 

Info

 messages are eventually discarded. This behavior is by design, to prevent your audit log from being cluttered with a mass of informational messages. In most instances, you are only interested in knowing when something goes wrong.

There are several cluster properties that can be used by advanced users to more precisely control policy message audits during the auditing process. Of particular importance are the following two properties:

audit.messageThreshold
audit.detailThreshold

REF: Audit Cluster Properties