Special characters decoding in HDFS command

Products

• Services Suite

Issue/Introduction

Symptoms:
You create a directory name with a special character in it. When running the HDFS ls command, the special character, such as # converting to %23 [ascii character], impacts the functionality of the application.

Environment

Cause

Creating a directory:

[gpadmin@hdm1 ~]$ hadoop fs -mkdir /tmp/10.10.11.44#11

drwxr-xr-x - gpadmin hadoop 0 2014-10-28 10:53 /tmp/10.10.11.44#11

Creating a directory inside the created directory: 

[gpadmin@hdm1 ~]$ hadoop fs -mkdir /tmp/10.10.11.44#11/hosts

Doing a listing on the directory structure using the HDFS command ls: 

[gpadmin@hdm1 ~]$ hadoop fs -ls /tmp/10.10.11.44#11

Found 1 items
drwxr-xr-x - gpadmin hadoop 0 2014-10-28 10:54 /tmp/10.10.11.44%2311/hosts

Resolution

Some of the special characters are not supported by the URI path in the Hadoop shell (usually you can replace # with !). It is recommended to use only characters which are mentioned in the JAVA URI.

http://docs.oracle.com/javase/7/docs/api/java/net/URI.html

Java can encode these paths correctly but not the Hadoop shell.