catalog issue gp_persistent_relation_node <=> filesystem still persists even after removing the extra files

Products

• VMware Tanzu Greenplum

Issue/Introduction

The catalog issue is like this:

20191124:10:22:15:123621 gpcheckcat:hebda-master-1:gpadmin-[ERROR]:master-1:5432:/dbdata1/master/gpseg-1
20191124:10:22:15:123621 gpcheckcat:master-1:gpadmin-[ERROR]:- tablespace_oid | database_oid | relfilenode_oid | segment_file_num | filesystem | persistent | relkind | relstorage
20191124:10:22:15:123621 gpcheckcat:master-1:gpadmin-[ERROR]:- 1663 | 61726 | 851744706 | 0 | t | f | None | None

Normally, to solve this issue, we would only need to move the file with relfilenode 851744706 in the database with oid 61726 in the master segment. However, sometimes we will find the gpcheckcat log still reported this issue with the same relfilenode_oid, like this:

20191124:14:53:19:186931 gpcheckcat:hebda-master-1:gpadmin-[ERROR]:-hebda-master-1:5432:/dbdata1/master/gpseg-1
20191124:14:53:19:186931 gpcheckcat:hebda-master-1:gpadmin-[ERROR]:- tablespace_oid | database_oid | relfilenode_oid | segment_file_num | filesystem | persistent | relkind | relstorage
20191124:14:53:19:186931 gpcheckcat:hebda-master-1:gpadmin-[ERROR]:- 62142 | 61726 | 851744706 | 0 | t | f | None | None

Environment

Product Version: 4.3

Resolution

The key point of this issue is the column tablespace_oid, which we may possibly ignore. In this case, the file that we have moved at the very beginning resides in the default tablespace. However, the current ERROR indicates a different tablespace_oid in which it has a file with the same relfilenode_oid.

To move such file, we can firstly locate the filespace of the tablespace 62142 using the sql below:

select fselocation from pg_filespace_entry where fsefsoid = (select spcfsoid from pg_tablespace where oid = 62142) and fsedbid = 1;

Note: Here, the column fsedbid refers to the dbid of master and the column fselocaton refers to the path of the filespace.
 

Then we can enter the directory of the filespace and move the extra file.