VMware Cloud Foundation /data partition where postgres saves data gets filled up

Products

• VMware Cloud Foundation

Issue/Introduction

Symptoms:

• SDDC Manager UI is not working.
• On restarting SDDC Manager, SDDC Manager services don't come up because of postgres related problems.
• postgres startup fails.
• The device mounted on filesystem mount point - /dev/mapper/data_vg-data - is full
•  

**Note this KB is applicable for VCF versions below 4.5**

For 4.5 and above - adjust the step for the /data partition with the steps from the below kb - see section "For SDDC Manager 4.5 and 5.0, you can follow the below steps."

SDDC Manager Appliance pre-check failing as root partition /dev/sda4 left with not enough space

https://knowledge.broadcom.com/external/article?articleNumber=316101

Environment

VMware Cloud Foundation 4.0.x

Resolution

Currently there is no resolution.

Workaround:

To workaround this issue, please follow the below steps:

1. Extend the /data partition by following these steps:

• Power off the SDDC Manager VM and wait for a few moments until it's properly turned off.

• Add a new disk to the VM as per below.

• Turn on the SDDC Manager VM and wait for it to properly come back up.
  
  
   

  2. Go to the SDDC Manager VM and run the following commands

•  echo "- - -" > /sys/class/scsi_host/host2/scan

•  lsblk

In your lsblk output you should see the new disk added.

In this example, I added a new disk with 16gb and it's denoted as sdg.

Also please note which disk your /data partition is located under. In this case mine is under sdc. As well as the path to the /data directory. In the screenshot mine is vg_data-lv_data1

3. Add the new disk

• pvcreate /dev/sdg      <---- Use the newly added disk

• Find out which Volume Group that your disk belongs to by running
  pvs

4. Extend the Volume Group with the New Disk

• vgextend vg_data /dev/sdg  <---- Use the newly added disk and VG

• lvdisplay data_vg ← In this case I'm looking for lv_data1 and I want to copy the path

Extend the space using the path

5.  lvextend -L+16G -r /dev/vg_data/lv_data1 . ← Might have to adjust the size to be less than 16G. Can specify what you want

6. You can verify with df -h that the /data directory now has grown in size.