"/storage" partition is 100% or Aria Suite Lifecycle 8.x upgrades fail due to lack of disk space

Products

• VMware Aria Suite

Issue/Introduction

• /storage partition on Aria Suite Lifecycle is 100% full. 
• The services on the appliance will not start and the UI is inaccessible.
• journalctl -x shows postgres service failures due to lack of available device space.

Environment

• VMware Aria Suite Lifecycle 8.x
• VMware vRealiez Suite Lifecycle Manager 8.x

Cause

The virtual machine appliance /storage partition is full.

Resolution

Note: The disk cannot be increased when a snapshot exists on the virtual machine. It is recommended to have daily backups of the system using any supported 3rd party utility that can manage vSphere snapshots as a backup provider before proceeding with these changes.

Procedure

1. Log in to Aria Suite Lifecycle via SSH as root.
2. Run the command df -h to list the disk usage of all partitions / disks.
   1. Using this information, isolate the disk and partition requiring additional space by running the below commands:
      lsblk

NAME                              MAJ:MIN RM  SIZE RO TYPE MOUNTPOINT
sda                                 8:0    0 10.6G  0 disk
├─sda1                              8:1    0    4M  0 part
├─sda2                              8:2    0   10M  0 part /boot/efi
├─sda3                              8:3    0  512M  0 part /boot
└─sda4                              8:4    0   10G  0 part
 └─system-system_0               254:5    0   10G  0 lvm  /
sdb                                 8:16   0 50.1G  0 disk
└─data-data_0                     254:4    0   50G  0 lvm  /data
sdc                                 8:32   0 10.1G  0 disk
└─storage-storage_0               254:3    0   10G  0 lvm  /storage
sdd                                 8:48   0  8.1G  0 disk
└─swap-swap_0                     254:2    0    8G  0 lvm  [SWAP]
sde                                 8:64   0 10.1G  0 disk
└─vg_alt_root-lv_alt_root         254:1    0   10G  0 lvm  /storage/alt_root
sdf                                 8:80   0  8.1G  0 disk
└─vg_lvm_snapshot-lv_lvm_snapshot 254:0    0    8G  0 lvm  /storage/lvm_snapshot
sr0                                11:0    1 1024M  0 rom

lvdisplay
 --- Physical volume ---
 PV Name               /dev/sdc
 VG Name               storage
 PV Size               10.06 GiB / not usable 4.00 MiB
 Allocatable           yes
 PE Size               4.00 MiB
 Total PE              2575
 Free PE               15
 Allocated PE          2560
 LV UUID               [UUID]

pvdisplay
--- Logical volume ---
 LV Path                /dev/storage/storage_0
 LV Name                storage_0
 VG Name                storage
 LV UUID                [UUID]
 LV Write Access        read/write
 LV Creation host, time photon, 2024-05-10 09:13:55 +0000
 LV Status              available
 # open                 1
 LV Size                10.00 GiB
 Current LE             2560
 Segments               1
 Allocation             inherit
 Read ahead sectors     auto
 - currently set to     256
 Block device           254:3
3. Shutdown Aria Suite Lifecycle Manager using a graceful Guest Operating system shut down.
4. Log in to vCenter and locate the Aria Suite Lifecycle virtual machine.
5. Right-click on the Aria Suite Lifecycle appliance and select Edit Settings.
6. Navigate to the Hard Disk section and increase the size (By 10Gb More) of the disk that corresponds to the full partition (e.g., Disk 3 for /storage).
7. Save the changes and power on the Aria Suite Lifecycle appliance.
8. When you reboot, the autogrow script detects the change in the logical volume and increases the size automatically.
   1. If the partitions are not updated after the reboot, manually run /etc/bootstrap/everyboot.d/20-autogrow-disk
9. Run df -h again to confirm that the storage size has been increased and there is now available space on the previously full partition.

If issues persist, follow instructions as outlined in Increase root '/' partition on VMware Aria Suite Lifecycle 8.x