How to configure HttpFS to access kerberized HDFS

Products

Services Suite

Issue/Introduction

This article discusses how to configure HttpFS to access HDFS secured with Kerberos.

Please refer to the procedure in this article if you want to access kerberized HDFS through HttpFS.

Environment

Resolution

1. Create httpFS and HTTP service principals on the KDC server:

# addprinc -randkey httpfs/<FQDN>@<REALM>
# addprinc -randkey HTTP/<FQDN>@<REALM>

Replace <FQDN> with a fully qualified domain name of the host where the HttpFS server is running.

Replace <REALM> with the name of your Kerberos realm.

In the examples in this article, FQDN will be admin.hadoop.local and REALM will be PIVOTAL.IO.

# kadmin.local -q "addprinc -randkey httpfs/[email protected] "
# kadmin.local -q "addprinc -randkey HTTP/[email protected]"

2. Create the keytab file with both principals:

# kadmin.local -q "ktadd -k /etc/security/keytabs/httpfs.service.keytab httpfs/[email protected] HTTP/[email protected]"

3. Distribute the keytab file to the machine that run HttpFS server. Normally put the file to /etc/security/keytabs/.

4. Change the ownership of keytab file to be owned by a user to run HttpFS server and group hadoop. And change permission of keytab file to 400.

5. Test that the keytab file works:

# kinit -kt /etc/security/keytabs/httpfs.service.keytab httpfs/[email protected]
[root@admin ~]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: httpfs/[email protected]

Valid starting Expires Service principal
12/01/15 08:33:47 12/02/15 08:33:47 krbtgt/[email protected]
 renew until 12/01/15 08:33:47

6. Add following properties to httpfs-site.xml. This is typically under /etc/hadoop-httpfs/conf.

<property>
  <name>httpfs.authentication.type</name>
  <value>kerberos</value>
 </property>
<property>
  <name>httpfs.hadoop.authentication.type</name>
  <value>kerberos</value>
 </property>
<property>
  <name>httpfs.authentication.kerberos.principal</name>
  <value>HTTP/[email protected]</value>
 </property>
<property>
  <name>httpfs.authentication.kerberos.keytab</name>
  <value>/etc/security/keytabs/httpfs.service.keytab</value>
 </property>
<property>
  <name>httpfs.hadoop.authentication.kerberos.principal</name>
  <value>httpfs/[email protected]</value>
 </property>
<property>
 <name>httpfs.hadoop.authentication.kerberos.keytab</name>
 <value>/etc/security/keytabs/httpfs.service.keytab</value>
 </property>
<property>
  <name>httpfs.authentication.kerberos.name.rules</name>
  <value>
        RULE:[2:$1@$0](rm@.*PIVOTAL.IO)s/.*/yarn/
        RULE:[2:$1@$0](nm@.*PIVOTAL.IO)s/.*/yarn/
        RULE:[2:$1@$0](nn@.*PIVOTAL.IO)s/.*/hdfs/
        RULE:[2:$1@$0](dn@.*PIVOTAL.IO)s/.*/hdfs/
        RULE:[2:$1@$0](hbase@.*PIVOTAL.IO)s/.*/hbase/
        RULE:[2:$1@$0](hbase@.*PIVOTAL.IO)s/.*/hbase/
        RULE:[2:$1@$0](oozie@.*PIVOTAL.IO)s/.*/oozie/
        RULE:[2:$1@$0](jhs@.*PIVOTAL.IO)s/.*/mapred/
        RULE:[2:$1@$0](jn/_HOST@.*PIVOTAL.IO)s/.*/hdfs/
        RULE:[2:$1@$0](falcon@.*PIVOTAL.IO)s/.*/falcon/
        DEFAULT
    </value>
</property>

Note:

Replace principal names and path of keytab files accordingly
Use the value configured for hadoop.security.auth_to_local in core-site.xml for httpfs.authentication.kerberos.name.rules

7. Restart HttpFS service for the configuration changes to take effect:

# service hadoop-httpfs restart

8. Test access to secured HDFS through HttpFS is successful. The following is a test to get Kerberos credential and list root directory in HDFS:

[root@admin ~]# kinit -kt /etc/security/keytabs/httpfs.service.keytab HTTP/[email protected]
[root@admin ~]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: HTTP/[email protected]
Valid starting Expires Service principal
12/01/15 06:53:15 12/02/15 06:53:15 krbtgt/[email protected]
 renew until 12/01/15 06:53:15
[root@admin ~]# curl --negotiate -i -L -u: 'http://admin.hadoop.local:14000/webhdfs/v1/?op=LISTSTATUS'
HTTP/1.1 401 Unauthorized
Server: Apache-Coyote/1.1
WWW-Authenticate: Negotiate
Set-Cookie: hadoop.auth=; Path=/; Expires=Thu, 01-Jan-1970 00:00:00 GMT; HttpOnly
Content-Type: text/html;charset=utf-8
Content-Length: 997
Date: Tue, 01 Dec 2015 11:53:23 GMT
HTTP/1.1 200 OK
Server: Apache-Coyote/1.1
Set-Cookie: hadoop.auth="u=HTTP&p=HTTP/[email protected]&t=kerberos&e=1449006803887&s=qeIsIBmD6POgBrciNHuMna2ifrY="; Path=/; Expires=Tue, 01-Dec-2015 21:53:23 GMT; HttpOnly
Content-Type: application/json
Transfer-Encoding: chunked
Date: Tue, 01 Dec 2015 11:53:23 GMT
{"FileStatuses":{"FileStatus":[{"pathSuffix":"app-logs","type":"DIRECTORY","length":0,"owner":"yarn","group":"hadoop","permission":"777","accessTime":0,"modificationTime":1443577258258,"blockSize":0,"replication":0},{"pathSuffix":"apps","type":"DIRECTORY","length":0,"owner":"hdfs","group":"hdfs","permission":"755","accessTime":0,"modificationTime":1442907202986,"blockSize":0,"replication":0},{"pathSuffix":"hawq_data","type":"DIRECTORY","length":0,"owner":"postgres","group":"gpadmin","permission":"755","accessTime":0,"modificationTime":1442987912402,"blockSize":0,"replication":0},{"pathSuffix":"mapred","type":"DIRECTORY","length":0,"owner":"mapred","group":"hdfs","permission":"755","accessTime":0,"modificationTime":1442907099113,"blockSize":0,"replication":0},{"pathSuffix":"mr-history","type":"DIRECTORY","length":0,"owner":"hdfs","group":"hdfs","permission":"755","accessTime":0,"modificationTime":1442907099121,"blockSize":0,"replication":0},{"pathSuffix":"phd","type":"DIRECTORY","length":0,"owner":"hdfs","group":"hdfs","permission":"755","accessTime":0,"modificationTime":1442907132464,"blockSize":0,"replication":0},{"pathSuffix":"system","type":"DIRECTORY","length":0,"owner":"hdfs","group":"hdfs","permission":"755","accessTime":0,"modificationTime":1442907074968,"blockSize":0,"replication":0},{"pathSuffix":"tmp","type":"DIRECTORY","length":0,"owner":"hdfs","group":"hdfs","permission":"777","accessTime":0,"modificationTime":1442990872079,"blockSize":0,"replication":0},{"pathSuffix":"user","type":"DIRECTORY","length":0,"owner":"hdfs","group":"hdfs","permission":"755","accessTime":0,"modificationTime":1443576819986,"blockSize":0,"replication":0}]}}

Additional Information

For additional information, refer to the following knowledge articles: