API gateway: Optimizing for Large Files and Messages

Products

• CA API Gateway

Issue/Introduction

Background

The default configuration of the CA API Gateway is optimized for handling messages in the range of 1 to 50 . There are a several specific changes that are required for tuning the Gateway appliance to handle larger messages.

 

 

 

Implementation

Several settings and configuration files must be changed in order to meet this use case:

1. The amount of system memory allocated to Java application.
2. Cluster-wide properties that negotiate various maximum data sizes.
3. Certain policy changes to limit message sizes.
4. Listen Ports

Environment

Release:
Component: APIGTW

Resolution

Java memory allocation

To check for how much memory is being allocated to java for the gateway, use the ps -efa | grep Gateway.jar | grep --color Xmx command.

The following configuration file has the multiplication factor for the amount of memory. By default its 1/2

/opt/SecureSpan/Gateway/runtime/etc/profile.d/appliancedefs.sh

Change the multiplication factor to a higher number than 1/2. Example 2/3 or 4/5

For more information on this please see: https://ca-broadcom.wolkenservicedesk.com/external/article?articleId=12306

Cluster-Wide Properties

Log in to the CA API Gateway Policy Manager as an administrative user.
Open the Manage Cluster-Wide Properties task
Add or edit the following properties:

1. io.signedPartMaxBytes = 0
2. io.xmlPartMaxBytes = 0
3. request.compress.gzip.allow = true
4. response.compress.gzip.allow = true

There are also three additional cluster-wide properties that need to be set with customized values:

1. template.partBodyMaxSize
2. attachment.diskThreshold
3. io.httpParamsMaxFormPostBytes

template.partBodyMaxSize specifies the maximum amount of memory to use for storing message parts. The value of this property should meet or exceed the maximum expected size of an individual message part. Setting this property to a value that exceeds the total size of the message should be sufficient.

attachment.diskThreshold specifies the maximum size of a message attachment to store in memory before stashing it to disk. If a message attachment exceeds the size of this value then the attachment will be stashed to disk. This value should meet or exceed the maximum total size of any individual message attachments for MTOM or Soap With Attachments.

io.httpParamsMaxFormPostBytes maximum number of bytes to buffer when processing an HTTP form POST.  Technical Note:  The io.httpParamsMaxFormPostBytes cluster property replaces the former com.l7tech.message.httpParamsMaxFormPost system property.  However if the system property is set, it overrides this cluster property.

Policy Changes

In some circumstances, it may be necessary to change one or more service policies or global policy fragments to minimize the risk of globally increasing the maximum message and attachment thresholds. There are a few factors that are at play with respect to this behavior and what should be changed. The following items must be taken into consideration:

1. The current value of io.xmlMaxPartBytes
2. The current use of the Limit Message Size assertion
3. Whether or not Perform WS-Security Processing is enabled for a particular service
4. Whether or not the service policy acts upon the request or response message

If the service policy acts upon a request or response message that exceeds io.xmlMaxPartBytes then the request will fail. "Acts upon" includes but is not limited to: Any threat protection assertion, XPath evaluation, regular expression evaluation, message transformation. This limitation is overridden by using Limit Message Size assertion. The Gateway will enforce the message size on a per-service basis as long as that assertion is used. The assertion is overridden by enabling "Perform WS-Security Processing..." within the Service Properties dialog.  If this is enabled then io.xmlMaxPartBytes overrides the value of Limit Message Size assertion. This occurs because the WS-Security processing specified above occurs before the service policy itself (and before Limit Message Size would take over as the authority).

Below are some examples to illustrate the interaction of these changes:

1. If a service policy streams the request or response and does not act upon the message then the Gateway does not limit the size of a message. Assertions are only necessary to shield a protected service from attack relying upon message size.
2. If a service policy performs message transformation or evaluation then the policy should use the Limit Message Size assertion to set the maximum size on a per-service basis.
3. If a Gateway will always handle large messages and inspect, evaluate, or transform the request or response--regardless of the published service being consumed--then?set io.xmlMaxPartBytes to an adequately large number.
4. If a service on the Gateway requires WS-Security processing then set io.xmlMaxPartBytes to an adequately large number.
5. If multiple services on the Gateway require WS-Security processing then publish a "pre-security" global policy fragment and add the Limit Message Size assertion.

Listen Ports

This is specific to incoming Gateway requests that would be passed through to a backend or processed by Gateway and stopping there.
Complete the [Advanced] tab of the Listen Port Properties as follows:
If you need to support large uploads (>2GB), select the "Override maximum message size" check box and specify a new limit or allow unlimited message size.