Increasing HCX Manager Disk Space for HCX Software Version 4.10 or Higher
search cancel

Increasing HCX Manager Disk Space for HCX Software Version 4.10 or Higher

book

Article ID: 373238

calendar_today

Updated On:

Products

VMware HCX

Issue/Introduction

This document provides instructions on how to increase HCX Manager disk space specifically related to Section A) of Knowledge Article 373010 "HCX - Bulk Migration & Replication Assisted vMotion (RAV) scalability guide - for HCX version 4.10 or higher"

Note: Please refer to the above Knowledge Article before making any changes.

Environment

VMware HCX

Resolution

Section 1) Identify whether HCX Manager disk partitioning is LVM or non-LVM based

Step 1: Login to HCX Manager SSH Console using 'admin' user.

Step 2: Switch to 'root' user.

Step 3: Run following commands on prompt.

# lsblk -l --output SIZE,TYPE,NAME,MOUNTPOINT | grep common

Step 4: Identify whether HCX Manager is LVM or non-LVM based on below responses from Step 2.

44.4G part sda6 /common      --- non-LVM based, follow Section 2
44.4G lvm  vg01-lv01 /common --- LVM based, follow Section 3

NOTE: for a non-LVM based system that is already scaled to 'medium' the response will be as below.

120G part sdb1 /common_ext

Section 2) Non-LVM based HCX Manager Disk Setup

Section 2A) Adding a new disk and common partition space for a Non-LVM based HCX Manager with a Default 64GB Disk Setting

Step 1: Add a 120GB or 300GB disk to both HCX managers based on the desired Medium or Large Scale Form Factor. Refer to Broadcom Knowledge Article Add new virtual Disk in HCX Manager.

Step 2: Setup the created disk on the HCX managers.

NOTE: the example below assumes the new disk that was created is “sdc”. Use the command ls /dev/sd* to determine what your new disk name is that was created in step1 and substitute that name for “sdc"

  • Create one partition on the new disk, in the example below the new disk name is sdc: 

                  fdisk /dev/sdc                                                                                                                                                                                                                                                                                                                                              
     Command (m for help): p                                                                                                            
     Command (m for help): n                                                                                                            
     Command action                                                                                                                                        
        e  extended                                                                                                                                                 
        p primary partition (1-4)                                                                                                                               
     p  ——> enter "p" to select that option                                                                                                                                       
     Partition number (1-4): 1    
     First cylinder (1-22400, default 1): <RETURN> ——> to use the default “1”
        Using default value 1
     Last cylinder or +size or +sizeM or +sizeK (1-22800, default 22800): <RETURN>  ——> to use the default max size value of the disk you created
     Command (m for help): w                                                                                                                      
   The partition table has been altered !   

                                                                                                                                                                                                                            

  • Create an ext3 file system on the new disk partition created on step 11. Since partition 1was created  on disk sdc the name used is sdc1

              mkfs -t ext3 /dev/sdc1

    Caution: Ensure the right device is selected as there is no undo.

 

  • Run the fdisk command to verify the existence of the newly created disk partition:

                   fdisk -l

 

  • Run this command to create a mount point for the new disk partition:

                  mkdir /common_ext

 

  • Run this command to display the current content of /etc/fstab:

                   cat /etc/fstab

 

  • Using a text editor, edit fstab and add this line to add the disk partition to /etc/fstab so that it is available across reboots:

                   /dev/sdc1 /common_ext ext3 rw,nosuid,nodev,exec,auto,nouser,async 1 2

         Note: Use Linux VI editor to edit/modify the fstab file:

          A. Press the ESC key for normal mode.
          B. Press "i" Key for insert mode.
          C. Press ":q!" keys to exit from the editor without saving a file.
          D. Press ":wq!" keys to save the updated file and exit from the editor.

 

  • Restart the Linux virtual machine (HCX Manager VM) and the new disk partition is now available.

 

  • Check if /common_ext has been mounted and has the correct type.

                    df -hT

 

Step 3: Stop HCX services as below for 4.11.2 and earlier:

systemctl stop postgresdb
systemctl stop zookeeper
systemctl stop kafka
systemctl stop app-engine
systemctl stop web-engine
systemctl stop appliance-management

for 4.11.3 and later:

systemctl stop mmd
systemctl stop hcmProbe
systemctl stop appliance-management
systemctl stop web-engine
systemctl stop plan-engine
systemctl stop app-engine
systemctl stop kafka
systemctl stop zookeeper
systemctl stop postgresdb

Step 4: Redirect existing contents under "kafka-db" and "postgres-db" to the newly created disk.

  • Move directory "/common/kafka-db" to "/common/kafka-db.bak".

cd  /common
mv kafka-db kafka-db.bak

  • Create a new directory "/common_ext/kafka-db".

cd  /common_ext
mkdir kafka-db

  • Note: The contents inside Kafka does not require being copied and will be generated after kafka/app-engine services restart.
  • Change the ownership and permissions of this directory same as "/common/kafka-db.bak".

chmod 755 kafka-db
chown kafka:kafka kafka-db

  • Make a soft link from "/common/kafka-db" to "/common_ext/kafka-db".

cd  /common
ln -s /common_ext/kafka-db kafka-db

  • Update the Kafka server properties to reflect the new location
    # vi /etc/kafka/server.properties
  • Find the line that says "log.dirs=" and change the right side to show "/common_ext/kafka-db"
  • Save the changes with "ESC wq!" 
  • Move directory "/common/postgres-db" to "/common/postgres-db.bak" as a backup.

cd  /common
mv postgres-db postgres-db.bak

  • Copy the content for directory "/common/postgres-db.bak" to "/common_ext/postgres-db" and change the ownership to postgres.
  • Note: Use "-R" option to change the ownership for "/common_ext/postgres-db" as below:

cp -r /common/postgres-db.bak /common_ext/postgres-db
chown -R postgres:postgres /common_ext/postgres-db

  • Make a soft link from "/common/postgres-db" to "/common_ext/postgres-db".

cd  /common
ln -s /common_ext/postgres-db postgres-db

Step 5: Start HCX services as below for 4.11.2 and earlier:

systemctl start postgresdb
systemctl start zookeeper
systemctl start kafka
systemctl start app-engine
systemctl start web-engine
systemctl start appliance-management

For 4.11.3 and later:

systemctl start postgresdb
systemctl start zookeeper
systemctl start kafka
systemctl start app-engine
systemctl start plan-engine
systemctl start web-engine
systemctl start appliance-management
systemctl start mmd
systemctl start hcmProbe

 

Section 2b) Increasing existing common_ext disk partition space for a NON-LVM based HCX Manager with a previously modified 300GB disk settings from Knowledge Article 321604 "HCX - Bulk Migration & Replication Assisted vMotion (RAV) scalability guide - for HCX version 4.7 to 4.9"

  • Verify existing disk space.

# fdisk -l | grep 'Disk /dev/sdb'

Disk /dev/sdb: 120 GiB, 128849018880 bytes, 251658240 sectors

  • Rescan.

# echo 1 > /sys/block/sdb/device/rescan

  • Verify.

# fdisk -l | grep 'Disk /dev/sdb'

Disk /dev/sdb: 300 GiB, 322122547200 bytes, 629145600 sectors

  • Resize /common_ext.

# fdisk /dev/sdb

Welcome to fdisk (util-linux 2.37.4).
Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.

This disk is currently in use - repartitioning is probably a bad idea.
It's recommended to umount all file systems, and swapoff all swap
partitions on this disk.

Command (m for help): d
Selected partition 1
Partition 1 has been deleted.

Command (m for help): n
Partition type
   p   primary (0 primary, 0 extended, 4 free)
   e   extended (container for logical partitions)

Select (default p):

Using default response p.
Partition number (1-4, default 1):
First sector (2048-629145599, default 2048):
Last sector, +/-sectors or +/-size{K,M,G,T,P} (2048-629145599, default 629145599):

Created a new partition 1 of type 'Linux' and of size 300 GiB.

Partition #1 contains a ext3 signature.
Do you want to remove the signature? [Y]es/[N]o: No

Command (m for help): w
The partition table has been altered.
Syncing disks.

# resize2fs /dev/sdb1


Section 3) LVM based HCX Manager Disk Setup

Section 3A) Adding a new disk and common partition space for an LVM based HCX Manager with a Default 64GB Disk Setting

Step 1: Add a 120GB or 300GB disk to both HCX managers based on the desired Medium or Large Scale Form Factor. Refer to Broadcom Knowledge Article Add new virtual Disk in HCX Manager. for creating a new virtual disk to an existing Linux virtual machine.

NOTE: Be sure to REBOOT the HCX Manager AFTER adding the new disk. Failing to do so will cause the new /sdb partition to not populate.


Step 2: Login to HCX Manager SSH Console using 'admin' user.
Step 3: Switch to 'root' user.
Step 4: Verify current partition with '/common' mount path.

# df -h

Filesystem             Size  Used Avail Use% Mounted on
/dev/root              7.6G  4.0G  3.3G  55% /
devtmpfs               5.9G     0  5.9G   0% /dev
tmpfs                  5.9G   64K  5.9G   1% /dev/shm
tmpfs                  2.4G  644K  2.4G   1% /run
tmpfs                  4.0M     0  4.0M   0% /sys/fs/cgroup
/dev/sda2               10M  2.0M  8.1M  20% /boot/efi
/dev/sda4              7.6G   92K  7.3G   1% /slot2
/dev/mapper/vg01-lv01   44G  3.0G   39G   8% /common
tmpfs                  1.2G     0  1.2G   0% /run/user/1000

Step 5: To rescan the disks on the HCX Manager execute the following command.

# echo 1 > /sys/block/sdb/device/rescan

Step 6: To create and add a partition to the existing '/common' partition using LVM execute the following commands.

  • pvcreate /dev/sdb
        Physical volume "/dev/sdb" successfully created.
  • vgextend vg01 /dev/sdb
        Volume group "vg01" successfully extended

  •  lvm lvextend -l +100%FREE /dev/vg01/lv01

    Size of logical volume vg01/lv01 changed from 44.39 GiB (11364 extents) to <164.39 GiB (42083 extents).
    Logical volume vg01/lv01 successfully resized.
  • resize2fs -p /dev/mapper/vg01-lv01
    resize2fs 1.46.5 (30-Dec-2021)
    Filesystem at /dev/mapper/vg01-lv01 is mounted on /common; on-line resizing required
    old_desc_blocks = 3, new_desc_blocks = 11
    The filesystem on /dev/mapper/vg01-lv01 is now 43092992 (4k) blocks long.

Step 7: Verify if the partition has been extended.

# df -h

Filesystem             Size  Used Avail Use% Mounted on
/dev/root              7.6G  4.0G  3.3G  55% /
devtmpfs               5.9G     0  5.9G   0% /dev
tmpfs                  5.9G   64K  5.9G   1% /dev/shm
tmpfs                  2.4G  652K  2.4G   1% /run
tmpfs                  4.0M     0  4.0M   0% /sys/fs/cgroup
/dev/sda2               10M  2.0M  8.1M  20% /boot/efi
/dev/sda4              7.6G   92K  7.3G   1% /slot2
/dev/mapper/vg01-lv01  162G  3.1G  152G   2% /common
tmpfs                  1.2G     0  1.2G   0% /run/user/1000

Section 3b) Increase existing common disk partition space for LVM based HCX Manager with a previously modified 300GB disk settings from Knowledge Article 321604 "HCX - Bulk Migration & Replication Assisted vMotion (RAV) scalability guide - for HCX version 4.7 to 4.9"

Step 1: Login to HCX Manager SSH Console using 'admin' user.
Step 2: Switch to 'root' user.
Step 3: Verify current partition with '/common' mount path.

# df -h

Filesystem             Size  Used Avail Use% Mounted on
/dev/root              7.7G  4.2G  3.2G  57% /
devtmpfs                12G     0   12G   0% /dev
tmpfs                   12G   64K   12G   1% /dev/shm
tmpfs                  4.7G  668K  4.7G   1% /run
tmpfs                  4.0M     0  4.0M   0% /sys/fs/cgroup
/dev/sda2               10M  2.0M  8.1M  20% /boot/efi
/dev/sda3              7.6G  4.0G  3.3G  56% /slot1
/dev/mapper/vg01-lv01  162G  6.0G  149G   4% /common
tmpfs                  2.4G     0  2.4G   0% /run/user/1000

Step 4: To rescan the disks on the HCX Manager execute the following command.

# echo 1 > /sys/block/sdb/device/rescan

Step 5: Verify increased disk space.

# fdisk -l | grep /dev/sdb
Disk /dev/sdb: 300 GiB, 322122547200 bytes, 629145600 sectors

Step 6: Increase Physical Volume.

# pvresize /dev/sdb

  Physical volume "/dev/sdb" changed
  1 physical volume(s) resized or updated / 0 physical volume(s) not resized

Step 7: Extend Volume Group.

# lvm lvextend -l +100%FREE /dev/vg01/lv01

  Size of logical volume vg01/lv01 changed from <164.39 GiB (42083 extents) to <344.39 GiB (88163 extents).
  Logical volume vg01/lv01 successfully resized.

Step 8: Resize partition space.

# resize2fs -p /dev/mapper/vg01-lv01

    resize2fs 1.46.5 (30-Dec-2021)
    Filesystem at /dev/mapper/vg01-lv01 is mounted on /common; on-line resizing required
    old_desc_blocks = 11, new_desc_blocks = 22
    The filesystem on /dev/mapper/vg01-lv01 is now 90278912 (4k) blocks long.

Step 9: Verify increase partition space.

# df -h

Filesystem             Size  Used Avail Use% Mounted on
/dev/root              7.7G  4.2G  3.2G  57% /
devtmpfs                12G     0   12G   0% /dev
tmpfs                   12G   64K   12G   1% /dev/shm
tmpfs                  4.7G  672K  4.7G   1% /run
tmpfs                  4.0M     0  4.0M   0% /sys/fs/cgroup
/dev/sda2               10M  2.0M  8.1M  20% /boot/efi
/dev/sda3              7.6G  4.0G  3.3G  56% /slot1
/dev/mapper/vg01-lv01  339G  6.0G  319G   2% /common
tmpfs                  2.4G     0  2.4G   0% /run/user/1000

 

 

Additional Information

There is no supported way to change an existing HCX Manager using LVM partitions to non-LVM partitions.
Bulk Migration & Replication Assisted vMotion (RAV) scalability guide - for HCX version 4.10 or higher

Powered by Wolken Software