Amazon AWS S3 returns 500 & 503 HTTP Error when sending data via Proxy

Products

• SG-S500
• ProxySG Software - SGOS
• ISG Proxy

Issue/Introduction

When sending application data to an Amazon S3 server through a network Proxy, connectivity issues may arise where the S3 bucket returns an AmazonS3Exception. These failures typically manifest as HTTP 500 Internal Error or HTTP 503 Slow Down responses visible in Syslog or packet traces. Even when the proxy policy is configured to allow all URLs for the Cloud Infrastructure category and communication appears successful in Wireshark, these errors can occur intermittently.

The 500 Internal Error indicates that the S3 service is unable to handle the request at that time, while the 503 Slow Down response typically signifies that the number of requests to the S3 bucket is exceeding the supported request rate or performance thresholds. These symptoms are generally related to origin server capacity rather than a block by the network proxy. Troubleshooting should focus on verifying the request frequency and bucket performance settings on the AWS side, as the proxy is simply observing and passing the "unavailable" status codes returned by the S3 service

Cause

The HTTP 500 and 503 errors are generated by the Amazon S3 origin server and are not caused by a block or misconfiguration on the network Proxy.

• HTTP 503 Slow Down: This occurs when the number of requests to the S3 bucket is very high and exceeds the supported request rate or performance thresholds of the bucket.
• HTTP 500 Internal Error: This indicates that the Amazon S3 service encountered an internal error and was unable to process the request at that specific time.

In these scenarios, the proxy correctly tunnels the application data and simply reports the "unavailable" or "internal error" status codes received back from the AWS infrastructure

Resolution

• Review the Syslog or packet traces to confirm the specific error code being returned by the Amazon S3 server (e.g., HTTP 500 or HTTP 503).
• Verify that the network proxy policy is correctly set to Allow for the required AWS domains and the Cloud Infrastructure category.
• If communication is successful in a packet capture but intermittently returns a 500 or 503 error, determine which specific error is occurring:
  • HTTP 500 Internal Error: This indicates Amazon S3 cannot handle the request at that time.
  • HTTP 503 Slow Down: This indicates the number of requests to the S3 bucket is exceeding the supported request rate.
• If the error is an HTTP 503 Slow Down, optimize the application's request rate to align with AWS performance guidelines. Refer to the Amazon S3 Request Rate and Performance Considerations for detailed limits.
• If the error is an HTTP 500 Internal Error, retry the request as this is typically a transient server-side issue.
• Confirm that the issue is not related to a proxy block, as these errors are generated by the origin server and observed/passed by the proxy.
• For additional interpretation of these errors, consult the AWS Knowledge Center for HTTP 5xx errors.