Failed to generate system log bundle error in VRMS Appliance
Article ID: 370011
Updated On:
Products
Issue/Introduction
1. Failed to generate system log bundles from the VRMS appliance (VAMI).
2. It's observed low free disk space on /dev/sda3
3. vSphere Replication plugin in vCenter showing 'Insufficient disk space'
Environment
vSphere Replication 8.x
vSphere Replication 9.x
Cause
1) Log collection fails due to space constraints or the log partition being full.
2) 'Insufficient disk space' warning for vSphere Replication on vCenter as VR appliance root partition is full.
Resolution
A. Follow the instructions to delete the old log entries in vSphere Replication plugin reports low disk space in vCenter
Even after deleting the old log files and still the disk space or log partition shows full then increase the disk space of the vSphere Replication by following the below steps.
B. Check and increase the size of the Hard Disk 1 disk of the vSphere Replication Appliance virtual machine
1. Shutdown the vSphere Replication Appliance VM2. Increase the virtual disk size of Hard Disk 1 to 20 GB.
4. Power ON the VM
5. Launch the vSphere Replication Appliance web console and run the following command: /opt/vmware/bin/extend_system_partition.sh
The following steps outline the procedure for manually increasing the partition size.
Check the current size of /opt/vmware/logs & /opt/vmware/support
root@VRMS [ ~ ]# df -hFilesystem Size Used Avail Use% Mounted ondevtmpfs 4.0M 0 4.0M 0% /devtmpfs 18G 0 18G 0% /dev/shmtmpfs 7.1G 8.8M 7.1G 1% /runtmpfs 4.0M 0 4.0M 0% /sys/fs/cgroup/dev/sda5 14G 3.0G 9.8G 24% /tmpfs 18G 800K 18G 1% /tmp/dev/mapper/support_vg-logs 975M 307M 617M 34% /opt/vmware/logs/dev/sda3 230M 48M 170M 22% /boot/dev/mapper/support_vg-support 2.0G 1.4G 427M 77% /opt/vmware/support/dev/mapper/support_vg-heapdump 6.9G 96K 6.5G 1% /opt/vmware/heapdump/dev/mapper/support_vg-vrmsdb 9.8G 5.0G 4.4G 54% /var/lib/vrmsdb/dev/sda2 10M 2.0M 8.1M 20% /boot/efitmpfs 3.6G 0 3.6G 0% /run/user/666
lvscan shows the current size of the partitions -
root@VRMS [ ~ ]# lvscan ACTIVE '/dev/support_vg/logs' [1.00 GiB] inherit ACTIVE '/dev/support_vg/support' [2.00 GiB] inherit ACTIVE '/dev/support_vg/heapdump' [7.00 GiB] inherit ACTIVE '/dev/support_vg/vrmsdb' [<10.00 GiB] inherit
Delete any existing snapshots on the VM and power it OFF. Edit the VM settings and increase the size of the second virtual disk to 8 GB.
Power ON the VM and run the pvresize command to expand the physical volume.
root@VRMS [ ~ ]# pvresize /dev/sdb Physical volume "/dev/sdb" changed 1 physical volume(s) resized or updated / 0 physical volume(s) not resized
Run the pvscan command to check review the extended size.
root@VRMS [ ~ ]# pvscan PV /dev/sdb VG support_vg lvm2 [<28.00 GiB / 8.00 GiB free] Total: 1 [<28.00 GiB] / in use: 1 [<28.00 GiB] / in no VG: 0 [0 ]
Run the lvextend command to allocate space to the following partitions.
root@VRMS [ ~ ]# lvextend -l +20%FREE -r /dev/support_vg/logs Size of logical volume support_vg/logs changed from 1.00 GiB (256 extents) to 2.60 GiB (666 extents). Logical volume support_vg/logs successfully resized.resize2fs 1.46.5 (30-Dec-2021)Filesystem at /dev/mapper/support_vg-logs is mounted on /opt/vmware/logs; on-line resizing requiredold_desc_blocks = 1, new_desc_blocks = 1The filesystem on /dev/mapper/support_vg-logs is now 681984 (4k) blocks long.
root@VRMS [ ~ ]# lvextend -l +80%FREE -r /dev/support_vg/support Size of logical volume support_vg/support changed from 2.00 GiB (512 extents) to 7.12 GiB (1823 extents). Logical volume support_vg/support successfully resized.resize2fs 1.46.5 (30-Dec-2021)Filesystem at /dev/mapper/support_vg-support is mounted on /opt/vmware/support; on-line resizing requiredold_desc_blocks = 1, new_desc_blocks = 1The filesystem on /dev/mapper/support_vg-support is now 1866752 (4k) blocks long.
Run the lvscan & lvdisplay commands to review the size of thesepartitions.
root@VRMS [ ~ ]# lvscan ACTIVE '/dev/support_vg/logs' [2.60 GiB] inherit ACTIVE '/dev/support_vg/support' [7.12 GiB] inherit ACTIVE '/dev/support_vg/heapdump' [7.00 GiB] inherit ACTIVE '/dev/support_vg/vrmsdb' [<10.00 GiB] inherit root@VRMS [ ~ ]# lvdisplay --- Logical volume --- LV Path /dev/support_vg/logs LV Name logs VG Name support_vg LV UUID c9MESd-Qfbc-KB0O-UWe2-C41m-53uJ-zZooaX LV Write Access read/write LV Creation host, time photon-machine, 2024-11-25 01:54:39 +0000 LV Status available # open 1 LV Size 2.60 GiB Current LE 666 Segments 2 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 254:0
--- Logical volume --- LV Path /dev/support_vg/support LV Name support VG Name support_vg LV UUID cIlrjo-z4lA-Q9Sq-RKhh-l2Yu-BDGE-9FcQ0I LV Write Access read/write LV Creation host, time photon-machine, 2024-11-25 01:54:39 +0000 LV Status available # open 1 LV Size 7.12 GiB Current LE 1823 Segments 2 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 254:1
Verify the availability size of the partitions.
root@VRMS [ ~ ]# df -hFilesystem Size Used Avail Use% Mounted ondevtmpfs 4.0M 0 4.0M 0% /devtmpfs 18G 0 18G 0% /dev/shmtmpfs 7.1G 8.8M 7.1G 1% /runtmpfs 4.0M 0 4.0M 0% /sys/fs/cgroup/dev/sda5 14G 3.0G 9.8G 24% /tmpfs 18G 164K 18G 1% /tmp/dev/sda3 230M 48M 170M 22% /boot/dev/mapper/support_vg-logs 2.6G 287M 2.2G 12% /opt/vmware/logs/dev/mapper/support_vg-support 7.0G 1.1G 5.7G 16% /opt/vmware/support/dev/mapper/support_vg-vrmsdb 9.8G 4.9G 4.4G 53% /var/lib/vrmsdb/dev/mapper/support_vg-heapdump 6.9G 96K 6.5G 1% /opt/vmware/heapdump/dev/sda2 10M 2.0M 8.1M 20% /boot/efitmpfs 3.6G 0 3.6G 0% /run/user/666
Feedback
