Error: Dependency failed for /sysroot or /storage/log on vCenter Server Appliance

Products

• VMware vCenter Server

Issue/Introduction

vCenter Server Appliance (VCSA) fails to boot and enters emergency mode following an unplanned power outage or storage disconnection. The console displays critical dependency failures for the root file system.

After a storage outage, vCenter appliance may fail to start with errors indicating a file system corruption, such as:

 

[FAILED] Failed to start File System on /dev/vg_root_0/lv_root_0.

[FAILED] Failed to start File System ##########-####-####-####-###########.

 

 

You may also see any of the below dependency errors:

[DEPEND] Dependency failed for /sysroot.
[DEPEND] Dependency failed for Initrd Root File System.
[DEPEND] Dependency failed for /storage/log.
[DEPEND] Dependency failed for Local File Systems. 
[DEPEND] Dependency failed for Relo*figuration from the Real Root.

 

When SSH or using the console from the ESXi host that the vCenter VM is running on, the below messages are displayed:

Generating "/run/initramfs/rdsosreport.txt"

Entering emergency mode. Exit the shell to continue.
Type "journalctl" to view system logs.
You might want to save "/run/initramfs/rdsosreport.txt" to a USB stick or /boot after mounting them and attach it to a bug report.

Give root password for maintenance (or press Control-D to continue):

Although this may imply that vCenter server is in maintenance mode, the above messages show that the vCenter VM has booted in Emergency mode.

Environment

• vCenter Server Appliance 6.x, 7.x, 8.x
• Photon OS

 

Cause

An abrupt power loss, host crash, or storage failure causes file system inconsistencies (orphaned inodes) on the virtual appliance disks. The boot sequence cannot verify the integrity of the partitions and stops to prevent data corruption.

 

• VCSA is stuck in emergency mode.
• Console displays: [FAILED] Failed to start File System Check on /dev/vg_root_0/lv_root0
• Console displays: [DEPEND] Dependency failed for /sysroot
• Console displays: [DEPEND] Dependency failed for Initrd Root File System

 

Resolution

1. Take a snapshot or backup of the vCenter Server Appliance.
   
   
2. Reboot the vCenter Server Appliance.
   
   
3. After the VCSA Photon OS starts, press the e key to enter the GNU GRUB Edit Menu.
   
   
4. Locate the line that begins with the word Linux. The line should look like the following screenshot:
   
   
   
5. If applicable, after consoleblank=0 remove:
   

   scsi_mod.scan=sync $systemd_cmdline fi\ps=1

    

6. Append the following line to the end of the linux line:
   

   fsck.repair=yes

   

   
   
   
7. Press F10 to continue booting.
   
   
8. Once the vCenter boots to the login screen, perform another reboot and the vCenter will be back on the network.

 

Note: If the issue persists, refer to Manually running a filesystem check (KB 338155) for manual repair steps.

Additional Information

For assistance, see Contact Support. Scroll to the bottom of the page and click on your respective region.