Cannot enlarge string buffer

Products

• VMware Tanzu Greenplum

Issue/Introduction

Symptoms:

When trying to create a string buffer greater than 1 GB in size, an error is logged.

Error Message

Cannot enlarge string buffer containing 1073349879 bytes by 700160 more bytes.

Environment

Cause

There is a hardcoded limit, MaxAllocSize, of 1 GB for the size of a string buffer in Greenplum. There is no guarantee that allocations smaller than MaxAllocSize will succeed. However, allocation requests larger than MaxAllocSize will be definitely be denied.

This is deliberately chosen to correspond to the limiting size of variable-length, varlena, objects under TOAST.

Resolution

This is not indicating an issue with the system, but with the data being processed.

To resolve this issue, you will need to determine what data is being read that is larger than 1 GB. You must process the data in a different manner to avoid objects greater than 1 GB.