Query Against gp_toolkit.gp_log_database Failed with "ERROR: missing data ..." or "ERROR: extra data ..."

Products

• VMware Tanzu Greenplum

Issue/Introduction

Symptoms:

A simple select from table gp_toolkit.gp_log_database will fail with an error.

Error Message:

This error message actually won't have any information about the segment and would instead indicate that the issue occurred on the master.

gpadmin=# SELECT * from gp_toolkit.gp_log_database limit 1;

ERROR: missing data for column "loguser" 
CONTEXT: External table __gp_log_master_ext, line 1 of execute:cat $GP_SEG_DATADIR/pg_log/*.csv: "2017-01-19 08:53:26.433647 EST|200005747|angdwp02|p344613|th-242456800|3.1.177.113|57306|2017-01-19 ..."

This error message which has clear information about which segment is having the problem could help you to locate the issue more easily:

gpadmin=# SELECT * from gp_toolkit.gp_log_database limit 1 ;

ERROR: extra data after last expected column (seg25 slice1 avw7gpdbsdwq4.gphd.local:1026 pid=46649) 
DETAIL: External table __gp_log_segment_ext, line 12599448 of execute:cat $GP_SEG_DATADIR/pg_log/*.csv: "2017-01-09 00:07:07.395008 EST|502639486|angdwq01|p469150|th1194202912|172.28.8.250|3725|2017-01-08 ..."

Environment

Cause

The error message of this issue is pretty straightforward about the error location but, we still can't be certain about what causes the error. An extra line or space, or something unusual, or any symptom like this could've caused it.

 

Resolution

From the pg_log directory, check all files ending with .csv. Pay special attention to some user customized file. Locate the file, remove that line, or rename that file.

It will take effect immediately if it's been resolved. Just try to query the table again.