gpbackup / gprestore : Plugin API version is inconsistent across segments

Products

• VMware Tanzu Greenplum

Issue/Introduction

The utilities gpbackup or gprestore report "Plugin API version is inconsistent across segments" error:
 

[CRITICAL]:-Plugin API version is inconsistent across segments; please reinstall plugin across segments on 0 segments.

Full logs: 

20220427:12:14:02 gprestore:gpadmin:tstmibigbbmdw01:161176-[DEBUG]:-Checking plugin api version on all hosts
20220427:12:14:02 gprestore:gpadmin:tstmibigbbmdw01:161176-[DEBUG]:-source /usr/local/greenplum-db-5.29.3/greenplum_path.sh && /usr/local/greenplum-db-5.29.3/bin/gpbackup_ddboost_plugin plugin_api_version
20220427:12:14:02 gprestore:gpadmin:tstmibigbbmdw01:161176-[DEBUG]:-Plugin /usr/local/greenplum-db-5.29.3/bin/gpbackup_ddboost_plugin on content ID 48 with API version 0.4.0 is not consistent with version on another segment
20220427:12:14:02 gprestore:gpadmin:tstmibigbbmdw01:161176-[CRITICAL]:-Plugin API version is inconsistent across segments; please reinstall plugin across segments on 0 segments. See /home/gpadmin/gpAdminLogs/gprestore_20220427.log for a complete list of errors.
github.com/greenplum-db/gp-common-go-libs/cluster.LogFatalClusterError
/tmp/build/3e49593f/go/pkg/mod/github.com/greenplum-db/[email protected]/cluster/cluster.go:398
github.com/greenplum-db/gpbackup/utils.(*PluginConfig).checkPluginAPIVersion
/tmp/build/3e49593f/go/src/github.com/greenplum-db/gpbackup/utils/plugin.go:131
github.com/greenplum-db/gpbackup/utils.(*PluginConfig).CheckPluginExistsOnAllHosts
/tmp/build/3e49593f/go/src/github.com/greenplum-db/gpbackup/utils/plugin.go:94
github.com/greenplum-db/gpbackup/restore.RecoverMetadataFilesUsingPlugin
/tmp/build/3e49593f/go/src/github.com/greenplum-db/gpbackup/restore/wrappers.go:190
github.com/greenplum-db/gpbackup/restore.DoSetup
/tmp/build/3e49593f/go/src/github.com/greenplum-db/gpbackup/restore/restore.go:81
main.main.func1
/tmp/build/3e49593f/go/src/github.com/greenplum-db/gpbackup/gprestore.go:22
github.com/spf13/cobra.(*Command).execute
/tmp/build/3e49593f/go/pkg/mod/github.com/spf13/[email protected]/command.go:860
github.com/spf13/cobra.(*Command).ExecuteC
/tmp/build/3e49593f/go/pkg/mod/github.com/spf13/[email protected]/command.go:974
github.com/spf13/cobra.(*Command).Execute
/tmp/build/3e49593f/go/pkg/mod/github.com/spf13/[email protected]/command.go:902
main.main
/tmp/build/3e49593f/go/src/github.com/greenplum-db/gpbackup/gprestore.go:27
runtime.main
/usr/local/go/src/runtime/proc.go:255
runtime.goexit
/usr/local/go/src/runtime/asm_amd64.s:1581
20220427:12:14:02 gprestore:gpadmin:tstmibigbbmdw01:161176-[DEBUG]:-Beginning cleanup
20220427:12:14:02 gprestore:gpadmin:tstmibigbbmdw01:161176-[DEBUG]:-Cleanup complete

Environment

Product Version: 5.28

Resolution

It is likely that the gpbackup and gprestore were installed in correctly, with maybe an error during installation.

As reported in the error it is best to reinstall the API's by reinstalling the gpbackup and gprestore package with the following steps: 

+ Check the packages installed 

gppkg -q --all

+ Delete the gpbackup and gprestore package.

gppkg --remove pivotal_greenplum_backup_restore-1.24.0

+ If there were errors, it may need to clean the installation with:

gppkg --clean

+ Then reinstall the package

gppkg -i pivotal_greenplum_backup_restore-1.24.0-gp5-rhel-x86_64.gppkg

Retry the backup and restore.