How to relocate yarn-site.xml (yarn.nodemanager.local.dirs) in an existing HD/PHD cluster

Products

• Services Suite

Issue/Introduction

Symptoms:
The mount points for yarn.nodemanager.local.dirs are running out of space and you are looking for a way to relocate to different mount points.

Environment

Resolution

1. Create new directories.

2. Grant the same permissions to those new directories to match with the old directories.

3. Change the parameter in yarn-site.xml (through icm_client or by manually editing it) and confirm that this is done across all Yarn node managers.

Before changing any parameters please make a backup of the current configuration of the HD cluster. If icm_client is being used, backup the configuration using the following command:

icm_client fetch-configuration -l CLUSTERNAME -o LOCALDIR 

4. If you've only used icm_client to change the configurations in the past then make sure you use icm_client to make the same changes here (with command listed below). Else you can manually edit the yarn-site.xml file directly.

icm_client reconfigure -l CLUSTERNAME -c LOCALDIR 

5. Restart Yarn.

icm_client restart -l CLUSTERNAME -s yarn 

6. Confirm that the new mount points are in use.

Changes can be made to yarn-site.xml for the yarn.nodemanager.local.dirs parameter on any node managers. Changes can only be made one at a time.

For rollback purposes, please make sure that you do not unmount the old directories until confirming that things are working properly for the new mount points.

IMPORTANT Note: Please make sure that the new parent and child directories have the same exact permissions as the old mount point directories.