Gateway 11 virtual appliance fails on reboot

Products

• CA API Gateway

Issue/Introduction

If we start the new V11 gateway over VMware Client over: "Starting guest OS" we see sometimes (in about 30% of tried starts) an error in Startup.

It seems the appliance has sometimes problems to find the devices /dev/sda1 and /dev/sda2 which comes with the new V11 OVA.

Environment

Gateway 11.x Virtual Appliance

Cause

The problem occurs when the disk layout has been changed and a additional disk was added to the default OVA image disk layout.

Resolution

The disk devices sda and sdb are assigned to the actual disks but this is not a persistent mapping.

Sometimes the disk are mapped incorrectly on startup  sda=disk1 but some times sda=disk2, and sdb=other disk.

Because our OVA only has 1 disk out of the box, there is only one device sda, the default fstab has no issue using

filesystem mounts /dev/sda2 = /boot and /dev/sda1 = /boot/efi in this case.

When a second disk is added, there will be device sdb and that sometimes breaks these 2 mounts.

The solution is to use persistent references in fstab such as UUID, which we have verified and could be fixed from both regular/emergency mode.

• From normal boot.
• Run blkid to get the UUID for /dev/sda1 and /dev/sda2.
• Edit /etc/fstab
• Replace /dev/sda1 with UUID=<uuid of /dev/sda1>, and replace /dev/sda2 with UUID=<uuid of /dev/sda2>
• Reboot the VM

                                           0       1

From emergency mode, initially mount -a fails and the disk assigned swapped in this scenario.    

• From emergency boot when the boot failed.
• Run blkid to get the UUID for /dev/sda1 and /dev/sda2.
• Edit /etc/fstab
• Replace /dev/sda1 with UUID=<uuid of /dev/sdb1>, and replace /dev/sda2 with UUID=<uuid of /dev/sdb2>   (Note: As the disks are assigned in the wrong sequence we swap the UUID to get them correct the next reboot )
• Reboot the VM

Example fstab entry using filesystem mount using UUID.

UUID="6b172556-a2e0-46db-ad1f-62312678dd64"  /boot           ext3    rw,suid,nodev,exec,auto,nouser,async,relatime,acl         0       2
UUID="6A2E-BB7B"                                                  /boot/efi       vfat     umask=0077  

Additional Information

In the techdocs there is a alternative solution to ensure the correct utilization of persistent volume mappings using labels .

https://techdocs.broadcom.com/us/en/ca-enterprise-software/layer7-api-management/api-gateway/11-0/install-configure-upgrade/upgrade-the-gateway/understand-gateway-patches/patch-an-appliance-gateway/increase-disk-space-in-virtual-appliance-using-lvm.html

See the section  "Customizing the Storage Layout".