How to recover from the exception "org.apache.geode.cache.DiskAccessException: For DiskStore: cluster_config: Unknown version ordinal xxx found when recovering Oplogs" while upgrading

Products

• VMware Tanzu Gemfire

Issue/Introduction

This articles covers how to recover when the locator does not start and throws the following exception during an upgrade:

org.apache.geode.cache.DiskAccessException: For DiskStore: cluster_config: Unknown version ordinal xxx found when recovering Oplogs

This error appears when a lower version locator tries to connect to a higher version locator.

Environment

Product Version: 9.10

Resolution

In order to get out of this situation, we need to refresh the cluster configuration. We can do that using the following procedure:

Note: You must have a saved cluster configuration or be able to run gfsh scripts to recreate your cluster configuration.  

1. Stop the cluster.

2. Remove the cluster_config directory from all the GemFire locators.

• In GemFire, it will be in the locator directory named: "ConfigDiskDir_xxx"
• In GemFire for VMs, it will be in "/var/vcap/store/gemfire-locator/*" of the locator directory.

3. Start the cluster.

4. Then import the cluster configuration or run the appropriate gfsh scripts.

Note: This issue can occur in any version of VMware Tanzu GemFire.