How to migrate from GemFire to VMware Tanzu GemFire

Products

• VMware Tanzu Gemfire

Issue/Introduction

In general, it is recommended to use VMware Tanzu GemFire for VMware Tanzu Application Service (TAS) for VMs deployed applications that use GemFire. PCC is built on top of GemFire. VMware Tanzu GemFire is TAS for VMs backing service implementation from VMware GemFire. TAS for VMs provides high availability for services like VMware Tanzu GemFire. VMware Tanzu GemFire is configured to be fault tolerant by default. It reduces the potential for data loss during a network segmentation outage by being configured to hold multiple copies of data across multiple TAS for VMs availability zones. This reduces the possibility that all multiple copies of a given entry will be on the losing side of a network split.

VMware GemFire can also be used by TAS for VMs applications as an external service, but without the benefits previously mentioned. The VMware GemFire cluster(s) will have to be managed by an administrator and or custom management scripts. Many of the needed custom and manual operations are included in by default.

This article will describe how to migrate from VMware GemFire to VMware Tanzu GemFire with detailed information and steps.

Resolution

1. Convert Cache.xml to cluster configuration (gfsh script)

VMware Tanzu Gemfire does not support the use of Cache XML based configurations. You must use the cluster configuration service to configure and manage VMware Tanzu Gemfire clusters. The configuration is created using the gfsh shell. The cluster configuration service helps to automate the deployment and operation of VMware Gemfire/VMware Tanzu GemFire.

A single gfsh file containing multiple cluster creation statements can be used to replace a cache.xml. See the following link for mapping cache.xml configurations to gfsh scripts: 
    https://docs.vmware.com/en/VMware-GemFire/10.0/gf/tools_modules-gfsh-chapter_overview.html 

2. Eliminate resource management

If the resource management for eviction is set in the cache.xml such as the following:

<resource-manager eviction-heap-percentage="80" critical-heap-percentage="95" />

Or if the resource management is set during cache server startup:

gfsh>start server --name=Server1 --eviction-heap-percentage=80

These should be removed. VMware Tanzu GemFire manages its own eviction heap percentage policies.

3. Convert to VMware Tanzu GemFire security

VMware GemFire supports a plug-in module for security. This plugin provides integrated security such that a single security interface is used for all components. This includes:

• VMware GemFire application clients
• JMX clients
• Pulse/REST endpoints
• Security Post-processing of region data
• Filtering/updating of returned data
• SSL communication
• Peer-to-peer, client, JMX, gateway senders and receivers, and HTTP connections

VMware Tanzu GemFire includes a default security plug-in implementation.

Note: Security implementation cannot be customized.

Also, see this link for ​application TLS configuration. VMware Tanzu Gemfire relies on a TAS for VMs addon called IPsec for securing data in transit.

Currently, VMware Tanzu Gemfire service instances are created with three default GemFire user roles for interacting with clusters:

• A cluster operator manages the VMware Gemfire cluster and can access region data.
• A developer can access region data.
• A gateway sender propagates region data to another VMware Tanzu Gemfire service instance.

All client apps, gfsh, and JMX clients must authenticate as one of these user roles to access the cluster. Each user role is given predefined permissions for cluster operations.

• The cluster operator role has CLUSTER:MANAGE, CLUSTER:WRITE, CLUSTER:READ, DATA:MANAGE, DATA:WRITE, and DATA:READ permissions.
• The developer role has CLUSTER:READ, DATA:WRITE, and DATA:READ permissions.
• The gateway sender role has DATA:WRITE permission.

4. Validate Disk Store

The latest version of VMware Tanzu Gemfire supports persistence and overflow of data using disk stores. Use gfsh over HTTPS to connect to the VMware Tanzu Gemfire service instance using the cluster operator credentials. Connecting gfsh via JMX is not supported.

Use relative paths for disk store directory instead of absolute paths.

create disk-store --name=<name-of-disk-store> --dir=<relative/path/to/diskstore/directory>

The relative path will be created within /var/vcap/store/gemfire-server/.

5. VMware Tanzu GemFire Monitoring

VMware Tanzu GemFire clusters and brokers stream logs containing service metrics into TAS for VMs Loggregator Firehose. You can use any tool that has a corresponding TAS for VMs nozzle to read and monitor these metrics in real time. This allows standard monitoring and logging tools (such as Prometheus and Grafana) to read and monitor these metrics in real time.

Refer to the following documentation for metrics and Key Performance Indicators (KPI(s)) that should be monitored and take the appropriate actions.

6. VMware GemFire to VMware Tanzu GemFire data migration

You can use the VMware GemFire export/import gfsh statements to move data set to VMware Tanzu GemFire. It is recommended to use parallel export from your existing VMware GemFire cluster to partition the data based on members.

export data --region=$region --dir=$BACKUP_DIR --member=$member --parallel=true

It is recommended to also use parallel import into VMware Tanzu Gemfire.

import data --region=$region --dir=$BACKUP_DIR --member=$member --parallel=true

Execute a rebalance command after data is imported into VMware Tanzu Gemfire.

7. Deploy custom code using gfsh deploy

VMware Tanzu GemFire does not expose changing the CLASSPATH of the VMware Tanzu GemFire members. Instead, use the gfsh deploy Jar command to deploy custom code.

This includes Functions, Domain/Data objects, Cache Writer/Loaders, Cache Listener, AsyncEventListener, PartitionListener, etc.

Example deploy Jar command.

deploy --jar=$1

Deployed jars are pushed to each member in the cluster and added to its CLASSPATH dynamically. The class can be referenced after the deploy command is executed.

For example, the following configures a cache listener on a “test” region after a Jar containing the class is deployed.

deploy --jar=/jars/exampleCacheListener.jar

create region --name=/test --cache-listener=my.exampleCacheListener --type=PARTITION

Note: Function classes that implement org.apache.geode.cache.execute.Function interface are automatically detected and registered. They will be available using the gfsh>list functions command.

8. Data Serialization

Note: PDX serialization is configured by default in the latest VMware Tanzu GemFire version. Using the configure pdx gfsh command is not supported. VMware Tanzu GemFire sets the PDX read serialized settings to true. All region get operations, queries of PDX data will return PdxInstance object. If you need to put PDX records into regions from custom server-side code (for example: Functions) the domain objects should implement org.apache.geode.pdx.PdxSerializable. The default PDX PDX serialization does not automatically convert Java objects to PDX.

Domain objects should implement org.apache.geode.DataSerializable interface if GemFire Data Serialization option is needed.

When Data Serialization or server-side code is used, it is also recommended to deploy the jars using the gfsh deploy jar command.