A VMFS volume is not available and the VMkernel log reports the error: Device size mismatch or Device shrank or Device Expanded

Products

• VMware vSphere ESXi

Issue/Introduction

• A VMFS volume is not available.
• The VMkernel log contains messages similar to:
  
  
  • WARNING: LVM: 1650: [vmhba1:0:2:1] Device size mismatch (actual 16777083 blocks, stored 16777088 blocks)
    
    
  • WARNING: LVM: 2446: [naa.6090a018d094045259bf048f8a007041:1] Device shrank (actual size 2147504812 blocks, stored size 2147512192 blocks)

Environment

VMware ESX 4.0.x
VMware ESXi 3.5.x Embedded
VMware ESX Server 3.0.x
VMware ESXi 4.0.x Embedded
VMware ESXi 4.0.x Installable
VMware ESX 4.1.x
VMware ESXi 4.1.x Embedded
VMware ESXi 4.1.x Installable
VMware ESXi 3.5.x Installable
VMware ESX Server 3.5.x

Resolution

This issues occurs when the partition table geometry does not match the size of the partition table when the VMFS volume was created.

To correct the partition size:

1. Identify the device that contains the bad partition table. Run the command:
   
   esxcfg-vmhbadevs | grep <vmhba device identified in the log>
   
   Using the example from the Symptoms section, the command and resulting sample output would be:
   
   # esxcfg-vmhbadevs | grep vmhba1:0:2
   vmhba1:0:2 /dev/sdf
   
   In ESX 3.5, the error message reported in /var/log/vmkernel looks like:
   
   WARNING: LVM: 1625: [vml. 0200000000600601601d311f00f06679650293de11524149442035:1] Device size mismatch (actual 16777083 blocks, stored 16777088 blocks)
   
   In ESX 4.x, the error message reported in /var/log/vmkernel looks like:
   
   WARNING: LVM: 2446: [naa.6090a018d094045259bf048f8a007041:1] Device shrank (actual size 16777083 blocks, stored size 16777088 blocks)
   
   This command provides the device and this sample output:
   
   # esxcfg-mpath -lv | grep 0200000000600601601d311f00f06679650293de11524149442035
   vmhba1:1:0 vml.0200000000600601601d311f00f06679650293de11524149442035 /dev/sdf (#####MB) has # paths and policy of XXXXX
   
   
2. Calculate the ending expected ending block of the partition table.
   
   VMFS volumes on ESX 3 and ESX 4 hosts, when created using the VMware Infrastructure or vSphere Client (connected to the server directly or through vCenter Server) are aligned to sector 128.
   
   VMFS volumes that were upgraded from an ESX 2 host or incorrectly created by hand on the command line are set to sector 63. The <starting offset> number is 128 (most likely) or 63.
   
   To calculate the correct ending block, use this formula: <Stored> + <Starting offset> - 1
   
   In this sample, the volume was originally created by an ESX 3 host using the VMware Infrastructure Client, so the starting sector is 128:
   
   16777088 + 128 - 1 = 16777215
   
   
3. Recreate the partition so match the calculated geometry using fdisk:
   1. Start fdisk on the device using -u to be able to set the starting block:
      
      fdisk -u /dev/sdX
      
      
   2. If the partition exists, remove it:
      
      Note: This does not delete the data inside the partition.
      
      
      1. Press d and press Enter.
      2. Press 1 and press Enter.
         
         
   3. Create the partition:
      1. Press n and press Enter.
      2. Press P to create primary partition.
      3. Enter the calculated value of the starting offset (in this sample, 128) and press Enter.
      4. Enter the calculated value of the ending block (in this sample, 16777215) and press Enter.
         
         
   4. Change the partition to type FB:
      
      Note:
      
      1. Press t and press Enter.
      2. Press 1 and press Enter.
         
         Note: Check the option carefully for the partition type for particular ESX/ESXi version. In ESXi 4.1, option 1 creates FAT32 partition Type. Therefore, select appropriate value for the fb [VMFS] partition type.
         
         
      3. Enter fb and press Enter.
         
         
         
   5. Save the changes and exit.
      
      Press w and press Enter.

If fdisk shows a negative number of sectors you can use the parted tool to create the partition:

partedUtil set <diskName> "<partitonNum> <startSector> <endSector> <type> <attribute>"

Using our sample values:

partedUtil set /dev/sdf "1 128 16777215 251 0"

251 is the decimal representation of fb, which is the partition type for VMFS.

Note: In some cases, you may not be able to modify the partition if the LUN is being accessed by virtual machine running on other hosts.

1. Perform a rescan to read in the new values using one of these methods:
   
   
   • From the ESX console, issue the command: vmkfstools -V
     
   • From the vSphere Client:
     1. Click on the ESX host in the inventory.
     2. Click the Configuration tab.
     3. Click Storage adapters.
     4. Rescan for new datastores. You can deselect Scan for New Storage Devices.