Extract complete data sets from region using gfsh without implementation of code

Products

• VMware Tanzu Gemfire

Issue/Introduction

For some VMware GemFire use cases, in order to monitor data or regions, it may be necessary to extract complete data sets in a human readable format from a region using gfsh queries rather than using the gfsh export functionality.

Implementing custom code would likely be a more effective way to achieve this. However, for DevOps or support teams, the more direct or preferred way is to use gfsh queries. This article will supplement the VMware GemFire documentation with regards to considerations for using gfsh queries when the region size is huge or the returned objects are complex.

Environment

Product Version: 9.8

Resolution

As a best practice, gfsh queries should be optimized whenever possible. However, in some use cases, it is not feasible to optimize the query, as users are trying to get all the data entries into human readable format from

VMware GemFire with using a gfsh query execution. A typical example is as follows:

gfsh>query --query="select * from /ABCD_RegionOneForPeopleInfo”

1. Due to the large number of items in the results, the following information is often returned as the output of the gfsh query script:

Computed ColSize=-2 Set RESULT_VIEWER to external. This uses the 'less' command (with horizontal scrolling) to see wider results

 For this scenario, as the message says "Set RESULT_VIEWER to external. This uses the 'less' command (with horizontal scrolling) to see wider results", it's more like a gfsh configuration setting issue. Since 'APP_RESULT_VIEWER' is set as “basic”, to fix it, it is necessary to change the 'APP_RESULT_VIEWER' to ''external' with the following gfsh command:

gfsh>set variable --name=APP_RESULT_VIEWER --value=“external”

 Once you have made the above change, we will be able to see direct results from the query.

2. If the result set is particularly large or complex, best practice is to select only a subset of columns instead of “select *”. Additionally, the LIMIT option can be used reduce the number of rows. Alternately, the gfsh environment variable ‘APP_FETCH_SIZE’ (Default value is 100.) might also be needed. 

a. check the current value of APP_FETCH_SIZE: 

gfsh>echo --string=${APP_FETCH_SIZE}

b. Change the value of APP_FETCH_SIZE as needed:

gfsh>set variable --name=APP_FETCH_SIZE --value=80000

c. Check out to make sure the APP_FETCH_SIZE has been changed successfully.

3. Instead of showing up all the results in gfsh terminals, an option here is redirect the output from gfsh terminal to a text file to monitor the results. An example gfsh script is as following:

query --query="select * from /ABCD_RegionOneForPeopleInfo LIMIT 10" --file=entrylist.txt

References:

• [1]  https://docs.vmware.com/en/VMware-GemFire/10.0/gf/tools_modules-gfsh-chapter_overview.html