Command Center Error: "External Table XXX Command Ended with SHELL TERMINATED by Signal SIGPIPE (13)"

Products

• VMware Tanzu Greenplum

Issue/Introduction

Symptoms:

• Greenplum Command Center/gpperfmon is not populating the data.
• The directory "$MASTER_DATA_DIR/gpperfmon/data" shows that there are no recently generated files.
• The master log shows that the command center is failing to update the gpperfmon database and returning this error message instead:

  2014-04-09 09:51:54.380644 MDT,"gpmon","gpperfmon",p20856,th-1663838000,"127.0.0.1","28368",2014-04-09 09:51:40 MDT,71905434,con739411,cmd2,seg-1,,dx1095848,x71905434,sx1,"ERROR","38000","external table __gp_log_master_ext command ended with SHELL TERMINATED by signal SIGPIPE (13)","Command: execute:cat $GP_SEG_DATADIR/pg_log/*.csv",,,,,,0,,"url.c",1404,

Environment

Cause

The command center uses gptoolkit tool to generate the data and the external table "__gp_log_master_ext" under gptoolkit schema fetches information from the master logs. The master logs here have some bad characters/line due to which the external table "__gp_log_master_ext" was unable to retrieve the information.

 

Resolution

To identify the bad character/line on the file, follow these steps:

1. Move all the files from pg_log to a separate directory.
2, Copy a group of 5 log files to pg_log.
3. Check the log file which has the bad character after converting it to the UTF8 format using this command:

iconv -c -f UTF8 -t UTF8 <bad_log_file> > <no_bad_char_file>

Example:

iconv -c -f UTF8 -t UTF8 gpdb-2013-06-22_100404.csv > gpdb-2013-06-22_100404.csv_nobad

4. When the bad file is found, you can check the bad character or line with the below procedure:

diff <bad_log_file> > <no_bad_char_file>

Example:

diff -i gpdb-2013-06-22_100404.csv gpdb-2013-06-22_100404.csv_nobad

5. Once bad line/character is found, remove the line from the file.
6. Then test if the external table works using (after connecting to "psql -d gpperfmon"):

select * from gptoolkit.__gp_log_master_ext;