PXF loading job failed with error " The DiskSpace quota of /xx/xxx is exceeded: " on Tanzu Greenplum

Products

• VMware Tanzu Greenplum

Issue/Introduction

PXF loading job failed everytime, errors from pxf-service.log are:

grep Disk pxf-service.log
Caused by: shaded.parquet.org.apache.thrift.transport.TTransportException: org.apache.hadoop.hdfs.protocol.DSQuotaExceededException: The DiskSpace quota of /xxx/xx is exceeded: quota = 2038357232914 B = 1.85 TB but diskspace consumed = 2038693987491 B = 1.85 TB
Caused by: org.apache.hadoop.hdfs.protocol.DSQuotaExceededException: The DiskSpace quota of /xxx/xx is exceeded: quota = 2038357232914 B = 1.85 TB but diskspace consumed = 2038693987491 B = 1.85 TB
Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.hdfs.protocol.DSQuotaExceededException): The DiskSpace quota of /xxx/xx is exceeded: quota = 2038357232914 B = 1.85 TB but diskspace consumed = 2038693987491 B = 1.85 TB
org.apache.hadoop.hdfs.protocol.DSQuotaExceededException: The DiskSpace quota of /xxx/xx is exceeded: quota = 2038357232914 B = 1.85 TB but diskspace consumed = 2038693987491 B = 1.85 TB 

Resolution

It is caused by hadoop system's diskspace quota reached for this query. Contacting the hadoop admin to enlarge the diskspace quota will fix this.