How to change the Master and Segment Data Directories in HAWQ or Pivotal HDB
search cancel

How to change the Master and Segment Data Directories in HAWQ or Pivotal HDB

book

Article ID: 295012

calendar_today

Updated On:

Products

Services Suite

Issue/Introduction

This article describes how to change the master and segment data directories in HAWQ or Pivotal HDB.

Environment


Resolution

The steps mentioned below are for changing the HAWQ master data directory path and all the HAWQ segment data directory paths and migrating their relevant metadata objects on all the cluster nodes.

These two parameters are locked from being changeable from Ambari, by default, after initial HAWQ install:
  • hawq_master_directory
  • hawq_segment_directory

So, additional steps are taken on the Ambari Server host to do this.
  • Test the following in your lab environment first:
Perform only the master directory update first. Test and validate it. Then move on to the segment data directories.
  • Use the Ambari Wizard to remove HAWQ Standby Master (if configured):
  1. Login to your cluster Ambari Dashboard
  2. Go to Services  and then to HAWQ
  3. Then, go to Service Actions and from there, go to Remove HAWQ Standby
  4. Go through the "Remove HAWQ Standby Wizard" to complete the Standby Master removal.
  • Back up all of your existing HAWQ data directories on their local filesystems:
​Confirm and identify your current HAWQ Master data directory in both Ambari and the local HAWQ Master host filesystem:
  1. Login to your cluster Ambari Dashboard and go to Services and then to HAWQ
  2. Select Config
  3. Search for hawq_master_directory
  4. Verify against your local HAWQ Master host filesystem
 
Backup your HAWQ Master Directory:
  1. Stop the HAWQ Service from Ambari
  2. Logout of Ambari web UI
     
Move the existing Master directory to its new location:
Example: 
- mv /data/hawq/master /data/hawq/master_new
Make the necessary changes in Ambari:
    Log in to your Ambari server host as the user running the Ambari Server.

    Backup the equivalent file for your HAWQ release: 
    - cd /var/lib/ambari-server/resources/common-services/HAWQ/2.0.0/configuration/
    - cp -p hawq-site.xml hawq-site.xml.bak
    Edit the existing hawq-site.xml from: 
    <property>
<name>hawq_master_directory</name>
<display-name>HAWQ Master Directory</display-name>
<value>/data/hawq/master</value>
<description>
The base path for the HAWQ master data directory. Multiple directories are not allowed.
The default is /data/hawq/master.
</description>
<value-attributes>
<type>directory</type>
<overridable>false</overridable>
<editable-only-at-install>true</editable-only-at-install>
</value-attributes>
<on-ambari-upgrade add="true"/>
</property>
    to: 
    <property>
<name>hawq_master_directory</name>
<display-name>HAWQ Master Directory</display-name>
<value>/data/hawq/master_new</value>
<description>
The base path for the HAWQ master data directory. Multiple directories are not allowed.
The default is /data/hawq/master.
</description>
<value-attributes>
<type>directory</type>
<overridable>true</overridable>
<editable-only-at-install>false</editable-only-at-install>
</value-attributes>
<on-ambari-upgrade add="true"/>
</property>
    Restart ambari-server process on the host using the command: 
     ambari-server restart
    • Change HAWQ Master directory path to your new path:
    Example: Change to your new path, /data/hawq/master_new
    1. Log in to Ambari UI and go to Services and then to HAWQ
    2. Select the Config
    3. Search for hawq_master_directory and update the "hawq_master_directory" value in Ambari
    4. Save changes
    5. Restart the HAWQ Service via Ambari
    6. Test and validate HAWQ 
    • For HAWQ Segments Directories:
    1. Log in to your cluster Ambari Dashboard and go to Services and then to HAWQ
    2. Select Config
    3. Search for hawq_segment_directory
    4. Verify all of your local HAWQ Segment host filesystems
    Backup and move all of your HAWQ Segment directories:
    1. Stop HAWQ Service from Ambari
    2. Logout of Ambari web UI
    Move all existing segment directories to their new location:
    Example, on every HAWQ Segment host: 
    - mv /data/hawq/segment /data/hawq/segment_new
    • Make the necessary changes in Ambari:
    Log in to your Ambari server host as the user running the Ambari Server.

    Backup the equivalent file for your HAWQ release: 
    - cd /var/lib/ambari-server/resources/common-services/HAWQ/2.0.0/configuration/
    - cp -p hawq-site.xml hawq-site.xml.bak2
     
    Edit hawq-site.xml from: 
    <property>
<name>hawq_segment_directory</name>
<display-name>HAWQ Segment Directory</display-name>
<value>/data/hawq/segment</value>
<description>
The base path for the HAWQ segment data directory. Multiple directories are not allowed.
The default is /data/hawq/segment.
</description>
<value-attributes>
<type>directory</type>
<overridable>false</overridable>
<editable-only-at-install>true</editable-only-at-install>
</value-attributes>
<on-ambari-upgrade add="true"/>
</property>
    to: 
    <property>
<name>hawq_segment_directory</name>
<display-name>HAWQ Segment Directory</display-name>
<value>/data/hawq/segment_new</value>
<description>
The base path for the HAWQ segment data directory. Multiple directories are not allowed.
The default is /data/hawq/segment.
</description>
<value-attributes>
<type>directory</type>
<overridable>true</overridable>
<editable-only-at-install>false</editable-only-at-install>
</value-attributes>
<on-ambari-upgrade add="true"/>
</property>
     
    Restart the ambari-server process on the host using the command: 
    ambari-server restart
    • Change HAWQ Segment directory path to your new path:
    Example: Change to your new path, 
    /data/hawq/segment_new
    1. Log in to Ambari UI and go to Services and then to HAWQ
    2. Select the Config
    3. Search for hawq_segment_directory
    4. Now you can update hawq_segment_directory value in Ambari.
    5. Save the changes
    6. Restart HAWQ Service via Ambari
    7. Test and validate HAWQ
    • Lock the Ambari parameters again from being overridable
    Log in to your Ambari server host and edit the file: 
     /var/lib/ambari-server/resources/common-services/HAWQ/2.0.0/configuration/hawq-site.xml
    Change from: 
    <property>
<name>hawq_master_directory</name>
<display-name>HAWQ Master Directory</display-name>
<value>/data/hawq/master_new</value>
<description>
The base path for the HAWQ master data directory. Multiple directories are not allowed.
The default is /data/hawq/master.
</description>
<value-attributes>
<type>directory</type>
<overridable>true</overridable>
<editable-only-at-install>false</editable-only-at-install>
</value-attributes>
<on-ambari-upgrade add="true"/>
</property>
    to: 
    <property>
<name>hawq_master_directory</name>
<display-name>HAWQ Master Directory</display-name>
<value>/data/hawq/master_new</value>
<description>
The base path for the HAWQ master data directory. Multiple directories are not allowed.
The default is /data/hawq/master.
</description>
<value-attributes>
<type>directory</type>
<overridable>false</overridable>
<editable-only-at-install>true</editable-only-at-install>
</value-attributes>
<on-ambari-upgrade add="true"/>
</property>
    Also, change from: 
    <property>
<name>hawq_segment_directory</name>
<display-name>HAWQ Segment Directory</display-name>
<value>/data/hawq/segment_new</value>
<description>
The base path for the HAWQ segment data directory. Multiple directories are not allowed.
The default is /data/hawq/segment.
</description>
<value-attributes>
<type>directory</type>
<overridable>true</overridable>
<editable-only-at-install>false</editable-only-at-install>
</value-attributes>
<on-ambari-upgrade add="true"/>
</property>
    to: 
    <property>
<name>hawq_segment_directory</name>
<display-name>HAWQ Segment Directory</display-name>
<value>/data/hawq/segment_new</value>
<description>
The base path for the HAWQ segment data directory. Multiple directories are not allowed.
The default is /data/hawq/segment.
</description>
<value-attributes>
<type>directory</type>
<overridable>false</overridable>
<editable-only-at-install>true</editable-only-at-install>
</value-attributes>
<on-ambari-upgrade add="true"/>
</property>
    • Restart the Ambari-server process one last time to implement: 
      ambari-server restart
     
    Note: You can migrate the HAWQ spill or temp directories via Ambari but only after performing similar file system backups and preparation.

    The following are the related parameters: 
    • hawq_master_temp_directory
    • hawq_segment_temp_directory


    Powered by Wolken Software