Unable to Expand VMFS Volume after increasing LVM Capacity on a local RAID

Products

• VMware
• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• After increasing the LVM capacity on a local RAID, the partition shows the correct size, but VMFS cannot be expanded to the complete capacity of the partition
• The disk capacity of a local LVM has been increased by adding more disks and expanding the LVM.
  
  
• When attempting to increase the VMFS size, no error is thrown.
  
  
• After the task to increase the VMFS completes, you can see that the partition now spans the set LVM size, however the VMFS capacity does not span the full partition size.
  
  
• In /var/log/vmkernel.log,you can see IncraseDatastoreWizard task:
  
  

  2017-03-28T06:59:17.477Z cpu27:68683 opID=72876b0f)World: 12230: VC opID IncreaseDatastoreWizard-apply-202528-ngc-75-cc-3234 maps to vmkernel opID 72876b0f

  2017-03-28T06:59:17.477Z cpu27:68683 opID=72876b0f)LVM: 4012: [naa.#############################:#] Device expanded (actual size 49211242463 blocks, stored size 21090531295 blocks)

  2017-03-28T06:59:17.541Z cpu27:68683 opID=72876b0f)LVM: 4012: [naa.#############################:#] Device expanded (actual size 49211242463 blocks, stored size 21090531295 blocks)

  2017-03-28T06:59:17.541Z cpu27:68683 opID=72876b0f)LVM: 14447: LVM device naa.#############################:# successfully expanded (new size: 25196156141056)

  2017-03-28T06:59:17.578Z cpu27:68683 opID=72876b0f)LVM: 10659: Using all available space (14397804118016).

  2017-03-28T06:59:17.585Z cpu27:68683 opID=72876b0f)LVM: 10569: Successfully added space (0) on device naa.#############################:# to volume 58a1c10c-18255b6e-0941-a0369fd9d7fc

  2017-03-28T06:59:17.927Z cpu27:68683 opID=72876b0f)Res6: 2111: 'example_datastore_1': resized LFBC cache from 20 to 32 buckets (1258 LFBCs to 2048 LFBCs)

Note: LVM device new size is higher than the size reported for "Using all available space"
  
  

• Before and after the IncreaseDatastoreWizard task you can see messages similar to:
  
  2017-03-28T06:58:32.181Z cpu##:######)lsi_mr3: megasas_hotplug_work:258: event code: 0x34.
  
  Note: this message is dependent on the controller and driver being used.

In this example, the event code 0x34 stands for "Inizialization in Progress".

Environment

VMware vCenter Server
VMware ESXi 

Resolution

When the initialization of the RAID in background has not completed yet, you experience this issue.

Expanding the partition queries the "max logical block address", where growing the VMFS volume queries "max usable logical block address".

For example, if the initialization is 50% complete, the VMFS capacity will show 50% of the partition size.

1. To grow the VMFS to the full partition capacity, wait until the RAID initialization completes.
   
   
2. Run the following command to determine the device name:
   
   vmkfstools -P "/vmfs/volumes/DatastoreName"
   
   Example output:
   
   VMFS-3.33 file system spanning 1 partitions.
File system label (if any): DatastoreName
Mode: public
Capacity 145223581696 (138496 file blocks * 1048576), 43937431552 (41902 blocks) avail
UUID: 4a14d968-88bf7161-700f-00145ef48f76
Partitions spanned (on "lvm"):
mpx.vmhba0:C0:T0:L0:3


3. Then run the below command:
   
   vmkfstools --growfs "/vmfs/devices/disks/Device:partition" "/vmfs/devices/disks/Device:partition"

   Example :The command would look similar to:
   
   vmkfstools --growfs "/vmfs/devices/disks/mpx.vmhba0:C0:T0:L0:3" "/vmfs/devices/disks/mpx.vmhba0:C0:T0:L0:3"

 

After performing a HBA / VMFS Rescan on the host, you should now see the VMFS expanded to the partition size.