How to confirm bucket distribution with gfsh query in GemFire
Article ID: 294055
Updated On:
Products
VMware Tanzu Gemfire
Issue/Introduction
When some members of a GemFire system crash or some disk stores are lost, you may need to check:
- If the Partition Region has enough primary buckets and/or redundant copies or not.
- The Buckets, if any, with low redundancy.
- The cacheservers where the buckets are located.
This article covers a simple way to achieve this goal using gfsh commands without needing to code any custom function execution services.
Resolution
1. You can use the following gfsh command to confirm the specified Partition region's bucketCount, primaryBucketCount, numBucketsWithoutRedundancy:
For example:
2. Use the following gfsh query to show the bucket distribution, and which cacheservers the buckets are located.
For example (Sample Region Name is exampleRegion):
3. When the cluster has Redundancy Zones configured for members, you can use the following gsfh query to show bucket distribution with Redundancy Zone info.
For example:
3. Use the following gfsh query to find buckets with low redundancy.
For example:
show metrics --region=/PartitionRegionName
For example:
gfsh>show metrics --region=/exampleRegion
Cluster-wide Region Metrics
Category | Metric | Value
--------- | ---------------------------- | -----
cluster | member count | 4
| region entry count | 24
region | lastModifiedTime | -1
| lastAccessedTime | -1
| missCount | -1
| hitCount | -1
| hitRatio | -1.0
| getsRate | 0.0
| putsRate | 0.0
| createsRate | 0.0
| destroyRate | 0.0
| putAllRate | 0.0
partition | putLocalRate | 0.0
| putRemoteRate | 0.0
| putRemoteLatency | 0
| putRemoteAvgLatency | 0
| bucketCount | 226
| primaryBucketCount | 113
| numBucketsWithoutRedundancy | 0
| totalBucketSize | 48
| averageBucketSize | 0
diskstore | totalEntriesOnlyOnDisk | 0
| diskReadsRate | 0.0
| diskWritesRate | 0.0
| totalDiskWriteInProgress | 0
| diskTaskWaiting | -1
callback | cacheWriterCallsAvgLatency | 0
| cacheListenerCallsAvgLatency | 0
eviction | lruEvictionRate | 0.0
| lruDestroyRate | 0.0
2. Use the following gfsh query to show the bucket distribution, and which cacheservers the buckets are located.
For example (Sample Region Name is exampleRegion):
gfsh>query --query="select distinct p.bucketId as bucketId,p.bucketAdvisor.primary.equals(o) as isPrimary, o.name from (select * from /exampleRegion.cache.getAllRegions r where r.fullPath='/exampleRegion') r, r.regionAdvisor.proxyBucketArray p, p.bucketOwners o order by bucketId,isPrimary desc limit 1000000" Result : true Rows : 226 bucketId | isPrimary | name -------- | --------- | ------------ 0 | true | cacheserver2 0 | false | cacheserver4 1 | true | cacheserver2 1 | false | cacheserver4 2 | true | cacheserver2 2 | false | cacheserver4 3 | true | cacheserver2 3 | false | cacheserver4 4 | true | cacheserver2 4 | false | cacheserver4 5 | true | cacheserver4 5 | false | cacheserver2 6 | true | cacheserver4 6 | false | cacheserver2 7 | true | cacheserver4 7 | false | cacheserver2 8 | true | cacheserver4 8 | false | cacheserver2 9 | true | cacheserver2 9 | false | cacheserver4 10 | true | cacheserver2 10 | false | cacheserver4 11 | true | cacheserver4 11 | false | cacheserver2 12 | true | cacheserver4 12 | false | cacheserver2 13 | true | cacheserver2 13 | false | cacheserver4 14 | true | cacheserver4 14 | false | cacheserver2 15 | true | cacheserver2 15 | false | cacheserver4 16 | true | cacheserver2 16 | false | cacheserver4 17 | true | cacheserver2 17 | false | cacheserver4 18 | true | cacheserver4 18 | false | cacheserver2 19 | true | cacheserver2 19 | false | cacheserver4 20 | true | cacheserver2 20 | false | cacheserver4 21 | true | cacheserver3 21 | false | cacheserver1 22 | true | cacheserver3 22 | false | cacheserver1 23 | true | cacheserver3 23 | false | cacheserver1 24 | true | cacheserver3 24 | false | cacheserver1 25 | true | cacheserver2 25 | false | cacheserver4 26 | true | cacheserver4 26 | false | cacheserver2 27 | true | cacheserver4 27 | false | cacheserver2 28 | true | cacheserver4 28 | false | cacheserver2 29 | true | cacheserver2 29 | false | cacheserver4 30 | true | cacheserver2 30 | false | cacheserver4 31 | true | cacheserver4 31 | false | cacheserver2 32 | true | cacheserver2 32 | false | cacheserver4 33 | true | cacheserver4 33 | false | cacheserver2 34 | true | cacheserver4 34 | false | cacheserver2 35 | true | cacheserver4 35 | false | cacheserver2 36 | true | cacheserver2 36 | false | cacheserver4 37 | true | cacheserver4 37 | false | cacheserver2 38 | true | cacheserver4 38 | false | cacheserver2 39 | true | cacheserver2 39 | false | cacheserver4 40 | true | cacheserver4 40 | false | cacheserver2 41 | true | cacheserver4 41 | false | cacheserver2 42 | true | cacheserver2 42 | false | cacheserver4 43 | true | cacheserver2 43 | false | cacheserver4 44 | true | cacheserver4 44 | false | cacheserver2 45 | true | cacheserver2 45 | false | cacheserver4 46 | true | cacheserver4 46 | false | cacheserver2 47 | true | cacheserver2 47 | false | cacheserver4 48 | true | cacheserver4 48 | false | cacheserver2 49 | true | cacheserver2 49 | false | cacheserver4 50 | true | cacheserver2 50 | false | cacheserver4 51 | true | cacheserver4 51 | false | cacheserver2 52 | true | cacheserver4 52 | false | cacheserver2 53 | true | cacheserver4 53 | false | cacheserver2 54 | true | cacheserver4 54 | false | cacheserver2 55 | true | cacheserver2 55 | false | cacheserver4 56 | true | cacheserver2 56 | false | cacheserver4 57 | true | cacheserver4 57 | false | cacheserver2 58 | true | cacheserver4 58 | false | cacheserver2 59 | true | cacheserver2 59 | false | cacheserver4 60 | true | cacheserver2 60 | false | cacheserver4 61 | true | cacheserver2 61 | false | cacheserver4 62 | true | cacheserver2 62 | false | cacheserver4 63 | true | cacheserver2 63 | false | cacheserver4 64 | true | cacheserver4 64 | false | cacheserver2 65 | true | cacheserver4 65 | false | cacheserver2 66 | true | cacheserver2 66 | false | cacheserver4 67 | true | cacheserver2 67 | false | cacheserver4 68 | true | cacheserver2 68 | false | cacheserver4 69 | true | cacheserver2 69 | false | cacheserver4 70 | true | cacheserver2 70 | false | cacheserver4 71 | true | cacheserver2 71 | false | cacheserver4 72 | true | cacheserver4 72 | false | cacheserver2 73 | true | cacheserver2 73 | false | cacheserver4 74 | true | cacheserver4 74 | false | cacheserver2 75 | true | cacheserver4 75 | false | cacheserver2 76 | true | cacheserver2 76 | false | cacheserver4 77 | true | cacheserver4 77 | false | cacheserver2 78 | true | cacheserver4 78 | false | cacheserver2 79 | true | cacheserver4 79 | false | cacheserver2 80 | true | cacheserver2 80 | false | cacheserver4 81 | true | cacheserver2 81 | false | cacheserver4 82 | true | cacheserver4 82 | false | cacheserver2 83 | true | cacheserver2 83 | false | cacheserver4 84 | true | cacheserver4 84 | false | cacheserver2 85 | true | cacheserver4 85 | false | cacheserver2 86 | true | cacheserver2 86 | false | cacheserver4 87 | true | cacheserver4 87 | false | cacheserver2 88 | true | cacheserver4 88 | false | cacheserver2 89 | true | cacheserver4 89 | false | cacheserver2 90 | true | cacheserver2 90 | false | cacheserver4 91 | true | cacheserver2 91 | false | cacheserver4 92 | true | cacheserver2 92 | false | cacheserver4 93 | true | cacheserver4 93 | false | cacheserver2 94 | true | cacheserver4 94 | false | cacheserver2 95 | true | cacheserver4 95 | false | cacheserver2 96 | true | cacheserver4 96 | false | cacheserver2 97 | true | cacheserver4 97 | false | cacheserver2 98 | true | cacheserver2 98 | false | cacheserver4 99 | true | cacheserver3 99 | false | cacheserver1 100 | true | cacheserver1 100 | false | cacheserver3 101 | true | cacheserver1 101 | false | cacheserver3 102 | true | cacheserver3 102 | false | cacheserver1 103 | true | cacheserver1 103 | false | cacheserver3 104 | true | cacheserver1 104 | false | cacheserver3 105 | true | cacheserver1 105 | false | cacheserver3 106 | true | cacheserver1 106 | false | cacheserver3 107 | true | cacheserver1 107 | false | cacheserver3 108 | true | cacheserver4 108 | false | cacheserver2 109 | true | cacheserver4 109 | false | cacheserver2 110 | true | cacheserver2 110 | false | cacheserver4 111 | true | cacheserver2 111 | false | cacheserver4 112 | true | cacheserver4 112 | false | cacheserver2
3. When the cluster has Redundancy Zones configured for members, you can use the following gsfh query to show bucket distribution with Redundancy Zone info.
For example:
gfsh>query --query="select distinct p.bucketId as bucketId,p.bucketAdvisor.primary.equals(o) as isPrimary,r.distributionManager.getRedundancyZone(o) as zone, o.name from (select * from /exampleRegion.cache.getAllRegions r where r.fullPath='/exampleRegion') r, r.regionAdvisor.proxyBucketArray p, p.bucketOwners o order by bucketId,isPrimary desc, zone desc limit 1000000"
Result : true
Rows : 226
bucketId | isPrimary | zone | name
-------- | --------- | ----- | ------------
0 | true | zone1 | cacheserver2
0 | false | zone2 | cacheserver4
1 | true | zone1 | cacheserver2
1 | false | zone2 | cacheserver4
2 | true | zone1 | cacheserver2
2 | false | zone2 | cacheserver4
3 | true | zone1 | cacheserver2
3 | false | zone2 | cacheserver4
4 | true | zone1 | cacheserver2
4 | false | zone2 | cacheserver4
5 | true | zone2 | cacheserver4
5 | false | zone1 | cacheserver2
6 | true | zone2 | cacheserver4
6 | false | zone1 | cacheserver2
7 | true | zone2 | cacheserver4
7 | false | zone1 | cacheserver2
8 | true | zone2 | cacheserver4
8 | false | zone1 | cacheserver2
9 | true | zone1 | cacheserver2
9 | false | zone2 | cacheserver4
10 | true | zone1 | cacheserver2
10 | false | zone2 | cacheserver4
11 | true | zone2 | cacheserver4
11 | false | zone1 | cacheserver2
12 | true | zone2 | cacheserver4
12 | false | zone1 | cacheserver2
13 | true | zone1 | cacheserver2
13 | false | zone2 | cacheserver4
14 | true | zone2 | cacheserver4
14 | false | zone1 | cacheserver2
15 | true | zone1 | cacheserver2
15 | false | zone2 | cacheserver4
16 | true | zone1 | cacheserver2
16 | false | zone2 | cacheserver4
17 | true | zone1 | cacheserver2
17 | false | zone2 | cacheserver4
18 | true | zone2 | cacheserver4
18 | false | zone1 | cacheserver2
19 | true | zone1 | cacheserver2
19 | false | zone2 | cacheserver4
20 | true | zone1 | cacheserver2
20 | false | zone2 | cacheserver4
21 | true | zone2 | cacheserver3
21 | false | zone1 | cacheserver1
22 | true | zone2 | cacheserver3
22 | false | zone1 | cacheserver1
23 | true | zone2 | cacheserver3
23 | false | zone1 | cacheserver1
24 | true | zone2 | cacheserver3
24 | false | zone1 | cacheserver1
25 | true | zone1 | cacheserver2
25 | false | zone2 | cacheserver4
26 | true | zone2 | cacheserver4
26 | false | zone1 | cacheserver2
27 | true | zone2 | cacheserver4
27 | false | zone1 | cacheserver2
28 | true | zone2 | cacheserver4
28 | false | zone1 | cacheserver2
29 | true | zone1 | cacheserver2
29 | false | zone2 | cacheserver4
30 | true | zone1 | cacheserver2
30 | false | zone2 | cacheserver4
31 | true | zone2 | cacheserver4
31 | false | zone1 | cacheserver2
32 | true | zone1 | cacheserver2
32 | false | zone2 | cacheserver4
33 | true | zone2 | cacheserver4
33 | false | zone1 | cacheserver2
34 | true | zone2 | cacheserver4
34 | false | zone1 | cacheserver2
35 | true | zone2 | cacheserver4
35 | false | zone1 | cacheserver2
36 | true | zone1 | cacheserver2
36 | false | zone2 | cacheserver4
37 | true | zone2 | cacheserver4
37 | false | zone1 | cacheserver2
38 | true | zone2 | cacheserver4
38 | false | zone1 | cacheserver2
39 | true | zone1 | cacheserver2
39 | false | zone2 | cacheserver4
40 | true | zone2 | cacheserver4
40 | false | zone1 | cacheserver2
41 | true | zone2 | cacheserver4
41 | false | zone1 | cacheserver2
42 | true | zone1 | cacheserver2
42 | false | zone2 | cacheserver4
43 | true | zone1 | cacheserver2
43 | false | zone2 | cacheserver4
44 | true | zone2 | cacheserver4
44 | false | zone1 | cacheserver2
45 | true | zone1 | cacheserver2
45 | false | zone2 | cacheserver4
46 | true | zone2 | cacheserver4
46 | false | zone1 | cacheserver2
47 | true | zone1 | cacheserver2
47 | false | zone2 | cacheserver4
48 | true | zone2 | cacheserver4
48 | false | zone1 | cacheserver2
49 | true | zone1 | cacheserver2
49 | false | zone2 | cacheserver4
50 | true | zone1 | cacheserver2
50 | false | zone2 | cacheserver4
51 | true | zone2 | cacheserver4
51 | false | zone1 | cacheserver2
52 | true | zone2 | cacheserver4
52 | false | zone1 | cacheserver2
53 | true | zone2 | cacheserver4
53 | false | zone1 | cacheserver2
54 | true | zone2 | cacheserver4
54 | false | zone1 | cacheserver2
55 | true | zone1 | cacheserver2
55 | false | zone2 | cacheserver4
56 | true | zone1 | cacheserver2
56 | false | zone2 | cacheserver4
57 | true | zone2 | cacheserver4
57 | false | zone1 | cacheserver2
58 | true | zone2 | cacheserver4
58 | false | zone1 | cacheserver2
59 | true | zone1 | cacheserver2
59 | false | zone2 | cacheserver4
60 | true | zone1 | cacheserver2
60 | false | zone2 | cacheserver4
61 | true | zone1 | cacheserver2
61 | false | zone2 | cacheserver4
62 | true | zone1 | cacheserver2
62 | false | zone2 | cacheserver4
63 | true | zone1 | cacheserver2
63 | false | zone2 | cacheserver4
64 | true | zone2 | cacheserver4
64 | false | zone1 | cacheserver2
65 | true | zone2 | cacheserver4
65 | false | zone1 | cacheserver2
66 | true | zone1 | cacheserver2
66 | false | zone2 | cacheserver4
67 | true | zone1 | cacheserver2
67 | false | zone2 | cacheserver4
68 | true | zone1 | cacheserver2
68 | false | zone2 | cacheserver4
69 | true | zone1 | cacheserver2
69 | false | zone2 | cacheserver4
70 | true | zone1 | cacheserver2
70 | false | zone2 | cacheserver4
71 | true | zone1 | cacheserver2
71 | false | zone2 | cacheserver4
72 | true | zone2 | cacheserver4
72 | false | zone1 | cacheserver2
73 | true | zone1 | cacheserver2
73 | false | zone2 | cacheserver4
74 | true | zone2 | cacheserver4
74 | false | zone1 | cacheserver2
75 | true | zone2 | cacheserver4
75 | false | zone1 | cacheserver2
76 | true | zone1 | cacheserver2
76 | false | zone2 | cacheserver4
77 | true | zone2 | cacheserver4
77 | false | zone1 | cacheserver2
78 | true | zone2 | cacheserver4
78 | false | zone1 | cacheserver2
79 | true | zone2 | cacheserver4
79 | false | zone1 | cacheserver2
80 | true | zone1 | cacheserver2
80 | false | zone2 | cacheserver4
81 | true | zone1 | cacheserver2
81 | false | zone2 | cacheserver4
82 | true | zone2 | cacheserver4
82 | false | zone1 | cacheserver2
83 | true | zone1 | cacheserver2
83 | false | zone2 | cacheserver4
84 | true | zone2 | cacheserver4
84 | false | zone1 | cacheserver2
85 | true | zone2 | cacheserver4
85 | false | zone1 | cacheserver2
86 | true | zone1 | cacheserver2
86 | false | zone2 | cacheserver4
87 | true | zone2 | cacheserver4
87 | false | zone1 | cacheserver2
88 | true | zone2 | cacheserver4
88 | false | zone1 | cacheserver2
89 | true | zone2 | cacheserver4
89 | false | zone1 | cacheserver2
90 | true | zone1 | cacheserver2
90 | false | zone2 | cacheserver4
91 | true | zone1 | cacheserver2
91 | false | zone2 | cacheserver4
92 | true | zone1 | cacheserver2
92 | false | zone2 | cacheserver4
93 | true | zone2 | cacheserver4
93 | false | zone1 | cacheserver2
94 | true | zone2 | cacheserver4
94 | false | zone1 | cacheserver2
95 | true | zone2 | cacheserver4
95 | false | zone1 | cacheserver2
96 | true | zone2 | cacheserver4
96 | false | zone1 | cacheserver2
97 | true | zone2 | cacheserver4
97 | false | zone1 | cacheserver2
98 | true | zone1 | cacheserver2
98 | false | zone2 | cacheserver4
99 | true | zone2 | cacheserver3
99 | false | zone1 | cacheserver1
100 | true | zone1 | cacheserver1
100 | false | zone2 | cacheserver3
101 | true | zone1 | cacheserver1
101 | false | zone2 | cacheserver3
102 | true | zone2 | cacheserver3
102 | false | zone1 | cacheserver1
103 | true | zone1 | cacheserver1
103 | false | zone2 | cacheserver3
104 | true | zone1 | cacheserver1
104 | false | zone2 | cacheserver3
105 | true | zone1 | cacheserver1
105 | false | zone2 | cacheserver3
106 | true | zone1 | cacheserver1
106 | false | zone2 | cacheserver3
107 | true | zone1 | cacheserver1
107 | false | zone2 | cacheserver3
108 | true | zone2 | cacheserver4
108 | false | zone1 | cacheserver2
109 | true | zone2 | cacheserver4
109 | false | zone1 | cacheserver2
110 | true | zone1 | cacheserver2
110 | false | zone2 | cacheserver4
111 | true | zone1 | cacheserver2
111 | false | zone2 | cacheserver4
112 | true | zone2 | cacheserver4
112 | false | zone1 | cacheserver2
3. Use the following gfsh query to find buckets with low redundancy.
For example:
gfsh>query --query="select distinct p.bucketId as bucketId,p.bucketAdvisor.primary.equals(o) as isPrimary,r.distributionManager.getRedundancyZone(o) as zone, o.name from (select * from /exampleRegion.cache.getAllRegions r where r.fullPath='/exampleRegion') r, r.regionAdvisor.proxyBucketArray p, p.bucketOwners o where p.bucketRedundancy < 1 order by bucketId,isPrimary limit 1000000"
Result : true
Rows : 100
bucketId | isPrimary | zone | name
-------- | --------- | ----- | ------------
0 | true | zone1 | cacheserver2
1 | true | zone1 | cacheserver2
2 | true | zone1 | cacheserver2
3 | true | zone1 | cacheserver2
4 | true | zone1 | cacheserver2
5 | true | zone1 | cacheserver2
6 | true | zone1 | cacheserver2
7 | true | zone1 | cacheserver2
8 | true | zone1 | cacheserver2
9 | true | zone1 | cacheserver2
10 | true | zone1 | cacheserver2
11 | true | zone1 | cacheserver2
12 | true | zone1 | cacheserver2
13 | true | zone1 | cacheserver2
14 | true | zone1 | cacheserver2
15 | true | zone1 | cacheserver2
16 | true | zone1 | cacheserver2
17 | true | zone1 | cacheserver2
18 | true | zone1 | cacheserver2
19 | true | zone1 | cacheserver2
20 | true | zone1 | cacheserver2
25 | true | zone1 | cacheserver2
26 | true | zone1 | cacheserver2
27 | true | zone1 | cacheserver2
28 | true | zone1 | cacheserver2
29 | true | zone1 | cacheserver2
30 | true | zone1 | cacheserver2
31 | true | zone1 | cacheserver2
32 | true | zone1 | cacheserver2
33 | true | zone1 | cacheserver2
34 | true | zone1 | cacheserver2
35 | true | zone1 | cacheserver2
36 | true | zone1 | cacheserver2
37 | true | zone1 | cacheserver2
38 | true | zone1 | cacheserver2
39 | true | zone1 | cacheserver2
40 | true | zone1 | cacheserver2
41 | true | zone1 | cacheserver2
42 | true | zone1 | cacheserver2
43 | true | zone1 | cacheserver2
44 | true | zone1 | cacheserver2
45 | true | zone1 | cacheserver2
46 | true | zone1 | cacheserver2
47 | true | zone1 | cacheserver2
48 | true | zone1 | cacheserver2
49 | true | zone1 | cacheserver2
50 | true | zone1 | cacheserver2
51 | true | zone1 | cacheserver2
52 | true | zone1 | cacheserver2
53 | true | zone1 | cacheserver2
54 | true | zone1 | cacheserver2
55 | true | zone1 | cacheserver2
56 | true | zone1 | cacheserver2
57 | true | zone1 | cacheserver2
58 | true | zone1 | cacheserver2
59 | true | zone1 | cacheserver2
60 | true | zone1 | cacheserver2
61 | true | zone1 | cacheserver2
62 | true | zone1 | cacheserver2
63 | true | zone1 | cacheserver2
64 | true | zone1 | cacheserver2
65 | true | zone1 | cacheserver2
66 | true | zone1 | cacheserver2
67 | true | zone1 | cacheserver2
68 | true | zone1 | cacheserver2
69 | true | zone1 | cacheserver2
70 | true | zone1 | cacheserver2
71 | true | zone1 | cacheserver2
72 | true | zone1 | cacheserver2
73 | true | zone1 | cacheserver2
74 | true | zone1 | cacheserver2
75 | true | zone1 | cacheserver2
76 | true | zone1 | cacheserver2
77 | true | zone1 | cacheserver2
78 | true | zone1 | cacheserver2
79 | true | zone1 | cacheserver2
80 | true | zone1 | cacheserver2
81 | true | zone1 | cacheserver2
82 | true | zone1 | cacheserver2
83 | true | zone1 | cacheserver2
84 | true | zone1 | cacheserver2
85 | true | zone1 | cacheserver2
86 | true | zone1 | cacheserver2
87 | true | zone1 | cacheserver2
88 | true | zone1 | cacheserver2
89 | true | zone1 | cacheserver2
90 | true | zone1 | cacheserver2
91 | true | zone1 | cacheserver2
92 | true | zone1 | cacheserver2
93 | true | zone1 | cacheserver2
94 | true | zone1 | cacheserver2
95 | true | zone1 | cacheserver2
96 | true | zone1 | cacheserver2
97 | true | zone1 | cacheserver2
98 | true | zone1 | cacheserver2
108 | true | zone1 | cacheserver2
109 | true | zone1 | cacheserver2
110 | true | zone1 | cacheserver2
111 | true | zone1 | cacheserver2
112 | true | zone1 | cacheserver2
Feedback
Yes
No
Powered by
