GemFire Locators don't start due to ConflictingPersistentDataException

Products

• VMware Tanzu Gemfire

Issue/Introduction

GemFire locators store cluster configuration information in internal, non-user created, persistent-replicated regions. Due to this, it is possible for a locator to encounter a ConflictingPersistentDataException on startup much like any cache node that hosts persistent regions. The solution is to determine which locator has the most correct cluster configuration data, usually the locator most recently running with the full cluster (e.g. the last one shutdown during a full cluster restart or an existing one when reintroducing a locator that has been down for some time). Once a "winner" (with the correct data) has been determined, all that is needed is to remove the Cluster Config disk store from the other and start it clean, so it can synchronize with the other locator(s).

Environment

Product Version: 10.x

Resolution

Looking at the error message, you will see it is pointing at a file path, something like this:

org.apache.geode.cache.persistence.ConflictingPersistentDataException: Region /_ConfigurationRegion remote member XXXX:7604:locator)<ec><v0>:41000 with persistent data /server_ip_address:/xxx/xxx/xxx/xxxx/ConfigDiskDir_xxxxx created at timestamp 1675657545168

On the "losing" (incorrect) locator host machine, navigate to the locator's working directory (typically where you start it) and find the ConfigDiskDir_xxxx directory. Remove this and the locator should successfully start and synchronize the current/correct Cluster Configuration with an existing locator.