Appliance booted into Emergency mode with Error: "Failed to start Switch Root" after attempting to apply 8.18 Hot Fix 1

Products

• VMware Aria Suite

Issue/Introduction

Primary node booted into Emergency mode.  

e2fsck -y on /dev/sda2 and /dev/sda4 are clean.

Environment

Aria Operations for Logs 8.18

Cause

Previous attempt to upgrade to Aria Operations for Logs 8.18 HF1 failed on the Primary node.  New Initrd image was added.  Primary node wasn't able to boot up properly on the new image as the environment is still on the old 8.18 platform.

From the above screenshot.  Aria Operations 8.18 has the initrd.img-5.10.216-2.ph4.  In addition, the new initrd.img-5.10-219-1.ph4 was added from the failed upgrade to HF1.  Grub was modified to point to the new image, causing the node to boot into Emergency mode.

Resolution

To fix the boot issue, the initial step is to boot up into the original Photon OS, instead of the Photon OS latest.  This enable grub to boot into the initrd.img-5.10.216-2.ph4 image.

Once the node has been booted up successfully, the next step is to remove the new image from /boot folder.  This free up space for /boot partition as it is configured with very limited space.  

• cd /boot
• rm initrd.img-5.10.219-1.ph4
• rm vmlinuz-5.10.219-1.ph4

Once the /boot disk space clean up has been completed, the next step is to perform the Aria Operations 8.18 HF1 manually on the Primary node via the following KB article:

KB344057 - How to Manually Upgrade VMware Aria Operations for Logs via Command Line

Once the upgrade is successful, the Primary node will reboot, and the UI will be accessible.

The subsequence upgrade on all the worker nodes will be automated.  The progress of the upgrade will be displayed on the Cluster status page of the Primary node UI.