vRealize Log Insight node failed to start file system

Products

• VCF Operations/Automation (formerly VMware Aria Suite)
• VMware vRealize Log Insight 8.x

Issue/Introduction

• vRealize Log Insight node failed to power on due to corruption with one of the filesystem and reports the following error as -
  

  [FAILED] Failed to start File Systemb-####-####-####-############
[DEPEND] Dependency failed for /sysroot.
[DEPEND] Dependency failed for Initrd Root File System.
[DEPEND] Dependency failed for Relomfiguration from the Real Root.
Generating "/run/initramfs/rdsosreport.txt"

  Entering emergency mode. Exit the shell to continue.
Type "journalctl" to view system logs.
You might want to save "/run/initramfs/rdsosreport.txt" to a USB stick or /boot
after mounting them and attach it to a bug report.

  Press Enter for maintenance
(or press Control-D to continue):





Environment

Aria Operations For Logs 8.18.x

Cause

This issue occurs when the virtual appliance experiences file inconsistencies after being forcefully halted as the result of a storage failure, power failure, or network outage.

Resolution

1. Before making any changes, take a powered off snapshot of vRLI VM or a backup
   
   
2. Run the following command and go to the last line (shift+G) and node the failed/corrupted file system uuid

   journalctl -xe

    

3. Now run the below fsck command against the same reported failed/corrupted file system and wait for it to fix all the broken inodes

   fsck -fy /dev/disk/by-uuid/<UUID>

4. Reboot the problem

   reboot -f

5. The appliance should now boot up and start all the necessary service, UI will show the node status as below:
   

Additional Information

In case the node fails to boot even after following the above action plan and reports error on /dev/sd# partition, kindly follow the below mentioned KB

Reference KB: Manual filesystem check for Aria Operations for Logs