Received error "invalid byte sequence for encoding ""UTF8"": 0x00" after upgrading to 6.x
search cancel

Received error "invalid byte sequence for encoding ""UTF8"": 0x00" after upgrading to 6.x

book

Article ID: 296834

calendar_today

Updated On: 04-22-2024

Products

VMware Tanzu Greenplum

Issue/Introduction

Following error is reported when loading data with the "COPY" command after upgrading to 6.21:
2023-01-05 14:38:13.112513 MST,"vmwares_db","test2",p44688,th161196160,"10.10.10.10","51832",2023-01-05 14:38:12 MST,0,con8193,cmd1,seg-1,,dx248674,,sx1,"ERROR","22021","invalid byte sequence for encoding ""UTF8"": 0x00",,,,,"COPY o_vmware_cust, line 1: ""2023-01-05^2022-12-12^NSB-3275993^5499479^2022-12-01^5957059^ACTIVE^Primary Borrower^Primary^C_CORP^...""","COPY hm_schema.vmware_cust FROM STDIN WITH DELIMITER E'^'",0,,"wchar.c",2011,


Environment

Product Version: 6.21

Resolution

It is a known issue that postgresql cannot handle 0x00 when using text formatter in GP6. We have created a text_in formatter to handle the 0x00 problem. But how to use text_in in copy is still under discussion(Jan 2023)

As a workaround for now R+D suggests using gpfdist or GPSS to load data from files.