Recovering RHEL 6 or CentOS 6 guest operating system from RAMDISK corruption

Products

VMware vSphere ESXi

Issue/Introduction

When you abruptly power off a RHEL 6 or CentOS 6 virtual machine without shutting down the virtual machine from operating system after you install, upgrade, or uninstall VMware Tools in a Linux environment (RHEL or CentOS 6), the guest operating system might fail during the next reboot due to a corrupted RAMDISK image file caused by incomplete write operation of the RAMDISK image to disk. An error similar to the following is reported in the console when you boot the guest operating system next time:

RAMDISK: incomplete write (31522 != 32768) write error Kernel panic - not syncing : VFS: Unable to mount root fs on unknown-block(0,0)

Environment

VMware vSphere ESXi 5.1
VMware vSphere ESXi 5.0
VMware vSphere ESXi 5.5

Resolution

This is not a VMware issue. For more information, consult your guest operating system vendor.

To resolve this issue, a Linux guest operating system with a corrupted RAMDISK image file is rescued to complete boot state by creating a new initramfs image in Linux rescue mode.

To create a new initramfs image and rescue the RHEL/CentOS 6 guest operating system from the corrupted RAMDISK image file, perform these steps:

Boot the virtual machine with the ISO image from which CentOS or RHEL guest is installed. The ISO installation process is described in Installing guest operating systems from ISO images (1002). To boot an existing VM from ISO, use steps 8 and 9 from that procedure.

Note: The preceding link was correct as of January 30, 2015. If you find the link is broken, provide a feedback and a VMware employee will update the link.
Select Rescue Installed System.
Select a language and key board type for this rescue, and then click Local CD/DVD as the rescue method.
Select No as the networking option, and then select Continue as the resume option to mount the actual root file system in read/write mode.
Mount the root file system that needs rescue to the location /mnt/sysimage and start a shell.
To change the root file system to actual root file system, run the command:

# chroot /mnt/sysimage
To identify the default kernel version and initramfs image from /boot/grub/grub.conf , run the command:

# cat /boot/grub/grub.conf For example, if the kernel version is 2.6.32-431.23.3.el6.x86_64, the output displayed is:

default=0 timeout=5 splashimage=(hd0,0)/grub/splash.xpm.gz hiddenmenu title Red Hat Enterprise Linux Server(2.6.32-431.23.3.el6.x86_64) root (hd0,0) kernel /vmlinuz-2.6.32-431.23.3.el6.x86_64 ro root=/dev/mapper/VolGroup-lv_root rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD rd_LVM_LV=VolGroup/lv_swap SYSFONT=latarcyrheb-sun16 crashkernel=128M rd_LVM_LV=VolGroup/lv_root KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb quiet initrd /initramfs-2.6.32-431.23.3.el6.x86_64.img title Red Hat Enterprise Linux (2.6.32-358.el6.x86_64) root (hd0,0) kernel /vmlinuz-2.6.32-358.el6.x86_64 ro root=/dev/mapper/VolGroup-lv_root rd_NO_LUKS LANG=en_US.UTF-8 rd_NO_MD rd_LVM_LV=VolGroup/lv_swap SYSFONT=latarcyrheb-sun16 crashkernel=128M rd_LVM_LV=VolGroup/lv_root KEYBOARDTYPE=pc KEYTABLE=us rd_NO_DM rhgb quiet initrd /initramfs-2.6.32-358.el6.x86_64.img
In above example, the default value 0 identifies the first title option to be used as default, the corresponding kernel version in title option line as 2.6.32-431.23.3.el6.x86_64 and the initramfs image file as /boot/initramfs-2.6.32-431.23.3.el6.x86_64.img.
After identifying the initramfs image, backup the original initramfs image before creating the new initramfs image by running the command:

# cp /boot/initramfs-2.6.32-431.23.3.el6.x86_64.img /boot/initramfs-2.6.32-431.23.3.el6.x86_64.img.ori
To create a new initramfs image, run the dracut command:

# /sbin/dracut -f /boot/initramfs-`uname -r`.img `uname -r`

For example, to create a new initramfs image file /boot/initramfs-2.6.32-431.23.3.el6.x86_64.img for kernel version 2.6.32-431.23.3.el6.x86_64, run the dracut command:

# /sbin/dracut -f /boot/initramfs-2.6.32-431.23.3.el6.x86_64.img 2.6.32-431.23.3.el6.x86_64
To reboot the system and verify that the guest boots successfully from RAMDISK image file, run the command:

# reboot

This issue is resolved in the following releases:

ESXi Version	Status
5.0	Fixed in VMware ESXi 5.0, Patch ESXi500-201408402-BG: Updates tools-light (2080819)
5.1	Fixed in VMware ESXi 5.1, Patch ESXi510-201503402-BG: Updates tools-light (2099295)
5.5	Fixed in VMware ESXi 5.5, Patch ESXi550-201509202-UG: Updates tools-light (2110233)

Additional Information

For translated versions of this article, see: