GFSH command "remove --region" does not support "--all" option for partitioned regions
Article ID: 294117
Updated On:
Products
Issue/Introduction
Symptoms:
GFSH does not support the gfsh> remove --region=/regionName --all operation for partitioned regions.
gfsh>remove --region=/part --all
The following error message is outputted:
gfsh>remove --region=/part --all Result : false Message : Option --all is not supported on partitioned region
Environment
Cause
gfsh> remove --region=/regionName --allfails on partitioned regions with: "Option --all is not supported on partitioned region." This is because --all maps to region.clear(), which partitioned regions don't support.
Resolution
There are two supported workarounds to remove entries in a controlled, batched way.
WORKAROUND 1: Client-side removal using keySetOnServer()
A small Java client program asks the servers for the FULL list of keys in the region (the whole region, not just one server's share), brings that list back to the client, then picks exactly how many keys to delete and tells the servers
to remove just those.
Where to run it: ONLY from a Java client program. There is no gfsh command for this. The actual deletes still happen on the servers. The client just decides which keys and how many.
Why use it: The number you ask for is the number that gets removed, for the WHOLE region, with no multiplication or guesswork about how many servers host the data. Best when you need an exact, predictable count removed (e.g. scripting this across many regions).
Trade-off: keySetOnServer() pulls the entire key list to the client in one call. For regions with a very large number of entries this has a real memory/network cost on the client (keys only, not values, so usually manageable).
Example:
Region<Object, Object> region = clientCache.getRegion("myRegion");
Set<Object> allKeys = region.keySetOnServer(); // one round trip; full key set
int limit = 5000; // exact total to remove, across the WHOLE region
int chunkSize = 500; // size of each removeAll() call
Set<Object> batch = new HashSet<>();
int removed = 0;
for (Object key : allKeys) {
if (removed >= limit) break;
batch.add(key);
removed++;
if (batch.size() == chunkSize || removed == limit) {
region.removeAll(batch);
batch.clear();
}
}Notes:
- Since the key list is a point-in-time snapshot (not backed live by the cache), it's safe to iterate and batch-remove.
- No server-side deployment needed -- simplest option if you're writing a small client utility.
WORKAROUND 2: Server-side custom Function using region.keySet()
Here, we can write a small Java function, deploy it to the Geode servers, and trigger it with a single gfsh command (or from client code). Each server that hosts part of the region grabs ITS OWN local entries and deletes up to the number you asked
for.
Where to run it: gfsh, or trigger it from client code -- either way, the deleting
happens on the servers.
IMPORTANT CAVEAT: the number you pass applies PER SERVER, not to the whole region.
If the region is spread across 2 servers and you ask for 4, 8 entries get removed (4 from each). Check how many members host the region with:
gfsh> describe region --name=/myRegionand multiply that member count by your requested number to know the real total.
Why use it: fastest to trigger (one gfsh command), no separate client program needed, good for quick, repeatable cleanup where "roughly N per server" is acceptable.
- Example Function code:
import org.apache.geode.cache.Declarable;
import org.apache.geode.cache.Region;
import org.apache.geode.cache.execute.Function;
import org.apache.geode.cache.execute.FunctionContext;
import org.apache.geode.cache.execute.FunctionException;
import org.apache.geode.cache.execute.RegionFunctionContext;
import org.apache.geode.cache.partition.PartitionRegionHelper;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import java.util.HashSet;
import java.util.Iterator;
import java.util.Set;
public class ClearRegionRemoveAllFunction implements Function, Declarable {
public static final String ID = "ClearRegionRemoveAllFunction";
private static final int DEFAULT_LIMIT = 100;
private static final int REMOVE_ALL_CHUNK_SIZE = 500;
private static final Logger logger = LogManager.getLogger(ClearRegionRemoveAllFunction.class);
private final java.util.function.Function<FunctionContext<?>, Region<Object, Object>> getRegion;
// No-arg constructor required so `gfsh deploy` can auto-register this function.
public ClearRegionRemoveAllFunction() {
this(ClearRegionRemoveAllFunction::defaultGetRegion);
}
public ClearRegionRemoveAllFunction(
java.util.function.Function<FunctionContext<?>, Region<Object, Object>> getRegion) {
this.getRegion = getRegion;
}
@Override
public void execute(FunctionContext functionContext) {
int limit = parseLimit(functionContext.getArguments());
if (limit <= 0) {
throw new IllegalArgumentException("limit must be > 0, got " + limit);
}
Region<Object, Object> region = getRegion.apply(functionContext);
// Snapshot keys first -- region.keySet() is a live view
Set<Object> keySet = new HashSet<>(region.keySet());
int localSize = keySet.size();
int toRemove = Math.min(limit, localSize);
logger.info("About to clear up to {} of {} local entries from region name={}; limit={}",
toRemove, localSize, region.getName(), limit);
int chunkSize = Math.min(toRemove, REMOVE_ALL_CHUNK_SIZE);
Set<Object> keysToClear = new HashSet<>(chunkSize);
int removed = 0;
Iterator<Object> iterator = keySet.iterator();
while (removed < toRemove && iterator.hasNext()) {
keysToClear.add(iterator.next());
if (keysToClear.size() == chunkSize || removed + keysToClear.size() == toRemove) {
region.removeAll(keysToClear);
removed += keysToClear.size();
keysToClear.clear();
}
}
logger.info("Cleared region name={}; removed={}; remainingLocalSize={}",
region.getName(), removed, region.size());
functionContext.getResultSender().lastResult(removed);
}
private int parseLimit(Object arguments) {
if (arguments == null) return DEFAULT_LIMIT;
if (arguments instanceof String[]) {
String[] args = (String[]) arguments;
return args.length == 0 ? DEFAULT_LIMIT : Integer.parseInt(args[0].trim());
}
if (arguments instanceof Number) return ((Number) arguments).intValue();
return Integer.parseInt(String.valueOf(arguments).trim());
}
@Override
public String getId() { return ID; }
@Override
public boolean optimizeForWrite() { return true; }
@Override
public boolean isHA() { return true; }
public static Region<Object, Object> defaultGetRegion(FunctionContext<?> functionContext) {
if (!(functionContext instanceof RegionFunctionContext)) {
throw new FunctionException("Execute this function onRegion");
}
RegionFunctionContext<?> rfc = (RegionFunctionContext<?>) functionContext;
Region<Object, Object> dataSet = rfc.getDataSet();
if (dataSet.getAttributes().getDataPolicy().withReplication()) {
return dataSet;
}
return PartitionRegionHelper.getLocalPrimaryData(rfc.getDataSet());
}
}- Deploy and register (once per cluster):
gfsh> deploy --jar=clear-region-function.jar- Run from gfsh:
gfsh> execute function --id=ClearRegionRemoveAllFunction --region=/myRegion --arguments=5000- Run from client code:
Region<Object, Object> region = clientCache.getRegion("myRegion");
List<Integer> results = FunctionService.onRegion(region)
.setArguments(new String[]{"5000"})
.execute(ClearRegionRemoveAllFunction.ID)
.getResult();Repeat the call (from gfsh or in a client-side loop, checking region.size()) until the
region is empty.
Feedback
