VMware ESXi datastore free space not updating after volume size increase from the vCenter Server

Products

• VMware vSphere ESXi

Issue/Introduction

This article provides information regarding issues where the ESXi fails to report the correct updated datastore capacity after a volume size increase in VMware vCenter Server 6.5.x and 6.7.x.

Symptoms:
After successfully increasing the volume size of a shared datastore from the vCenter Server 6.5.x and 6.7.x, you experience these symptoms:

• Free space on the shared datastore fails to update on some of the ESXi hosts.
• Shared datastore shows incorrect size on some of the ESXi hosts.
• A storage vMotion to another destination datastore fails.
• You see this error similar to:
  
  Insufficient space on datastore
   
• In the /var/log/vmkernel.log file of the ESXi host that reports the correct capacity of the shared datastore, you see entries similar to:
  
  [YYYY-MM-DDTHH:MM:SS] cpu12:77810 opID=9f937b12)World: 12230: VC opID IncreaseDatastoreWizard-apply-25137678-ngc:70327908-41-a9-eba3 maps to vmkernel opID 9f937b12
  [YYYY-MM-DDTHH:MM:SS] cpu12:77810 opID=9f937b12)LVM: 4014: [naa.xxxxxxxxxxxxxxxxxxxxx17:1] Device expanded (actual size 1572861919 blocks, stored size 1048573919 blocks)
  [YYYY-MM-DDTHH:MM:SS] cpu12:77810 opID=9f937b12)LVM: 4014: [naa.xxxxxxxxxxxxxxxxxxxxx:1] Device expanded (actual size 1572861919 blocks, stored size 1048573919 blocks)
  [YYYY-MM-DDTHH:MM:SS] cpu12:77810 opID=9f937b12)LVM: 14448: LVM device naa.xxxxxxxxxxxxxxxxxxxxx:1 successfully expanded (new size: 805305302528)
  [YYYY-MM-DDTHH:MM:SS] cpu12:77810 opID=9f937b12)Vol3: 1006: Unable to register file system Test for APD timeout notifications: Already exists
  [YYYY-MM-DDTHH:MM:SS] cpu12:77810 opID=9f937b12)LVM: 10661: Using all available space (268435456000).
  [YYYY-MM-DDTHH:MM:SS] cpu0:77810 opID=9f937b12)LVM: 10571: Successfully added space (0) on device naa.xxxxxxxxxxxxxxxxxxxxx :1 to volume 59fb71d0-f5ba54ed-79e7-f8bc12179f90

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Environment

VMware vSphere ESXi 6.7
VMware vSphere ESXi 6.5

Cause

This issue occurs due a slow refresh call being made across all the ESXi hosts after an extend datastore operations from the vCenter Server successfully completes.

Resolution

This issue is resolved in:

• VMware ESXi650-202007001 (Build 16576891), available at Broadcom Downloads.
• VMware ESXi 6.7 Update 1, available at Broadcom Downloads.

Workaround:
To work around this issue if you do not want to upgrade, VMware recommends to perform a cluster level rescan after expanding or increasing a datastore in the vCenter Server 6.5.x and 6.7.x. This forces the changes to all ESXi hosts that accesses the newly increased or expanded shared datastore.

Alternatively, run a Rescan Storage from the vSphere Client for each of the ESXi host not seeing the datastore size increase.

1. Log in as root on the ESXi host.
2. Check the datastore usage on the ESXi host by running this command:
   
   df -h
    
3. Refresh the VMFS volume by running this command:
   
   vmkfstools -v
   
   Note: Perform this to all the ESXi host connected to the same shared datastore.