How to safely clear a replicated region by using removeAll on batches of keys

Products

• VMware Tanzu Gemfire

Issue/Introduction

The issue is that server(s) may run out of memory when using removeAll to completely clear replicated regions that are configured with custom expiry, such as in the example below:

create region --name=regionXY --type=REPLICATE --entry-idle-time-expiration=3600 --entry-idle-time-expiration-action=destroy --entry-time-to-live-expiration=3600 --entry-time-to-live-expiration-action=destroy --enable-statistics

When such regions are cleared either using the @cacheEvict annotation or from gfsh as shown below, a memory leak will occur because both the RegionEntries and EntryExpiryTasks are still live as entries are being loaded into the region concurrent with the remove operation:

remove --region=value [--key=value] [--all(=value)?] 

Environment

Product Version: 9.9

Resolution

The fix for this issue will be in GemFire v. 9.9.3 , 9.10.3 and newer. As a workaround, we can clear the region using removeAll on smaller batches of keys. Below is a function that would clear the region:

import org.apache.geode.cache.Declarable;
import org.apache.geode.cache.Region;
​
import org.apache.geode.cache.execute.Function;
import org.apache.geode.cache.execute.FunctionContext;
import org.apache.geode.cache.execute.RegionFunctionContext;
​
import java.util.HashSet;
import java.util.Set;
​
// Note: This function works around GEM-3001 / GEODE-6564
public class ClearRegionRemoveAllFunction implements Function, Declarable {
​
  public void execute(FunctionContext context) {
    RegionFunctionContext rfc = (RegionFunctionContext) context;
    String[] arguments = (String[]) context.getArguments();
    int batchSize = Integer.parseInt(arguments[0]);
    Region region = rfc.getDataSet();
    
    // Destroy each entry using removeAll
    context.getCache().getLogger().info("About to clear region name=" + region.getName() + "; size=" + region.size() + "; batchSize=" + batchSize);
    long start=0, end=0;
    start = System.currentTimeMillis();
    Set keysToClear = new HashSet();
    int i = 0;
    for (Object key : region.keySet()) {
      keysToClear.add(key);
      if ((i+1) % batchSize == 0) {
        region.removeAll(keysToClear);
        keysToClear.clear();
      }
      i++;
    }
    region.removeAll(keysToClear);
    end = System.currentTimeMillis();
    context.getCache().getLogger().info("Cleared region name=" + region.getName() + "; size=" + region.size() + " in " + (end-start) + " ms");
​
    context.getResultSender().lastResult(true);
  }
​
  public String getId() {
    return getClass().getSimpleName();
  }
}

Please follow the steps below to delete the data from the region:

• Deploy the jar of above function code from gfsh as shown:

gfsh > deploy --jar=/Users/grawat/funcTest/target/funcTest-1.0-SNAPSHOT.jar

• Verify that the function is deployed using:

 gfsh > list functions

• Execute the function whenever you want to clear the data from a replicated region, using the following:

gfsh > execute function --id=ClearRegionRemoveAllFunction --region=data --arguments=12343