In Tanzu GemFire, Members cannot share disk-store directories, so colocated members using shared configurations must use relative paths.
Article ID: 294000
Updated On:
Products
Issue/Introduction
VMware GemFire cluster members cannot share disk store directories, which means that any members colocated on the same host must have distinct disk store directory paths.
When disk stores are configured via API or through separate, per-node cache.xml files, ensuring this separation is typically straightforward. However, in environments using shared configurations, such as disk store creation via GFSH or the Cluster Configuration Service, additional care must be taken, as these tools apply identical settings across all members.
This article outlines the key considerations and options for addressing this configuration requirement.
Cause
If multiple colocated members attempt to use the same disk store directory, only the first member to start will succeed. Subsequent members will fail to acquire the necessary file locks and encounter an error such as:
This occurs because GemFire locks the disk store files to prevent concurrent access by multiple JVMs. Therefore, using a shared path among colocated members results in a conflict over the lock file (*.lk).
Resolution
Options:
In such situations, there are a couple of options:
- Configure disk store directories using relative paths, rather than full paths, so that the disk store directories will be located relative to the member's working directory; or
- Use logical member groups (see reference below) to break members into groups so that colocated members are associated with distinct groups and then use different disk store configurations in each
group.
For example, in a simple cluster with one locator and two colocated cache servers:
gfsh>list members Name | Id -------- | --------------------------------------------------------------- locator1 | 10.0.0.10(locator1:19579:locator)<ec><v0>:1024 [Coordinator] server2 | 10.0.0.10(server2:20561)<v1>:1025 server3 | 10.0.0.10(server3:20752)<v1>:1026
If we attempt to create a disk store using the gfsh 'create disk-store' command, the disk store will be successfully on the first but the second will error:
gfsh>create disk-store --name=DiskStore --dir=/home/gemfire/DiskStore Member | Status ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ server2 | Success server3 | ERROR: org.apache.geode.cache.DiskAccessException: For DiskStore: DiskStore Could not lock "/home/gemfire/DiskStore/DRLK_DiskStore.lk". Other JVMs might have created diskstore with same name.
If, however, we use a relative path (i.e. not specified from the root directory) both will succeed:
gfsh>create disk-store --name=DiskStore --dir=/home/gemfire/DiskStore Member | Status ------- | -------- server2 | Success server3 | Success
Alternately, we could place each cache server in a separate group, using the VMware Gemfire 'groups' property and then create using the group parameter in the 'create disk-store' command:
gfsh>create disk-store --name=DiskStore --dir=/home/gemfire/DiskStore1 --group=group1 Member | Status ------- | -------- server2 | Success gfsh>create disk-store --name=DiskStore --dir=/home/gemfire/DiskStore2 --group=group2 Member | Status ------- | -------- server3 | Success
Additional Information
Feedback
