Calculating the Size of a GemFire Region Using ObjectGraphSizer

Products

• Pivotal GemFire
• VMware Tanzu Gemfire
• VMware Tanzu Greenplum / Gemfire
• VMware Tanzu Data Suite
• VMware Tanzu Data Suite
• VMware Tanzu Data Intelligence

Issue/Introduction

The purpose of this KB is to find out how much memory a GemFire Region is using for capacity planning or performance analysis.

The statistic PartitionedRegionStats.dataStoreBytesInUse helps estimate memory usage for partitioned Regions but doesn’t exist for replicated Regions. To get an accurate size for any Region, you can use the ObjectGraphSizer tool.

Environment

• GemFire (all versions)
• Works for both replicated and partitioned Regions

Cause

GemFire Regions store large numbers of objects in memory. Each Region (replicated or partitioned) has its own internal structure.

Without filtering, ObjectGraphSizer tries to measure every object in the JVM because Regions have references to many system objects like the cache and distributed system. This can cause incorrect or very large size results.

Resolution

You can accurately calculate Region size by combining two components:

• CalculateRegionSizeFunction – runs on the server to measure the Region size.

• RegionObjectFilter – filters out unrelated objects so only Region data is counted.

Steps to Implement

• Create the Function

The function gets the Region and uses ObjectGraphSizer to measure it:

private SingleMemberRegionSize calculateSize(Cache cache, Region region) {
  ObjectGraphSizer.ObjectFilter filter = new RegionObjectFilter(cache);
  long regionSize = ObjectGraphSizer.size(region, filter, false);
  String histogram = ObjectGraphSizer.histogram(region, filter, false);
  return new SingleMemberRegionSize(region.getName(), regionSize, histogram);
}

This returns:

• • Region name
  • Region size in bytes
  • A histogram in a SingleMemberRegionSize showing memory used by each object type

•  Create the Object Filter

The filter accepts or rejects objects during the measurement. It rejects common non-Region objects like caches, loggers, and system managers:   

public boolean accept(Object parent, Object object) {
  if (object instanceof Cache ||
      object instanceof DistributedSystem ||
      object instanceof Logger ||
      object instanceof PartitionedRegionStats ||
      object instanceof DiskStore) {
    return false;
  }
  return true;
}

   

• Run the Function

When you run the function, each GemFire member (server) calculates:

• • The total Region size (in bytes)
  • A histogram showing the number and size of each class type

Example client log output:

Member sizes for regions [ReplicatedTrade, PartitionedTrade]:
member=server-1 regionName=ReplicatedTrade; size=12,404,384 bytes
member=server-1 regionName=PartitionedTrade; size=9,716,936 bytes

4. View Histogram on Server

The server log shows details of object types and their memory usage:

Histogram for PartitionedTrade:
class java.util.concurrent.atomic.AtomicLong       33624    1401
class org.apache.geode.internal.cache.BucketRegion 39600    75
class [B                                           7344032  6701

Additional Information

All source code described in this article as well as an example usage is available here.