PXF accessing Azure HDFS with ADLS gen2

Products

• VMware Tanzu Greenplum

Issue/Introduction

The following error was reported when trying to connect to Azure Data Lake Storage (ADLS) gen2:

Exception java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.azurebfs.AzureBlobFileSystem not found

The following error was reported when trying to connect to ADLS gen2:

Configuration property host01.dfs.core.windows.net not found

Environment

Product Version: 6.13

Resolution

For the first error, the Hadoop 3.x java libraries are required to provide the java class. Either upgrade or install the hdfs client on the hosts in the Greenplum cluster, or add the required jar files to ${PXF_CONF}/lib directory. Then run "Configuration property host01.dfs.core.windows.net not found" to copy the files to all hosts in the cluster.

For the second error, the possible issues causing this error are:
 

• The access key is not specified in the servers xml config file
• The incorrect URL is specified in the LOCATION

See Hadoop Azure Support: ABFS — Azure Data Lake Storage Gen2 for more details.