Extending or increasing a datastore through vCenter Server fails
search cancel

Extending or increasing a datastore through vCenter Server fails

book

Article ID: 314373

calendar_today

Updated On:

Products

VMware vCenter Server VMware vSphere ESXi

Issue/Introduction

This article provides steps to resolve an issue where the added disk space is not visible on the vsphere client, or the datastore capacity is empty after extending or expanding a datastore through the vCenter Server using the vSphere Client.


Symptoms:

When trying to extend or increase the datastore from the VMware vCenter Server using the vSphere Client, you experience these symptoms:
  • The extend or increase datastore task fails fails to display the available unused disk space capacity.
  • After extending the LUN from the SAN, the new storage appears under Storage Adapters when viewed from the vCenter Server. However, the increased disk space is not showing or not available through the datastore properties.
  • The expanded or increased datastore capacity is empty.

Validation:

  • The vCenter Server applies filters to stop any possible data corruption on already used LUNs by other host.
  • You can validate these from the UI and the vpxd logs with these steps:
    • Review LUN IDs for the LUNs in question in the UI on all hosts to make sure they are consistent.
    • Enable verbose logging for the vpxd logs and attempt to expand/increase the datastore. For more information, see the Set Verbose Logging section of the vSphere Monitoring and Performance Guide.
    • In the vpxd.log file, you see entries similar to: 

      [QueryAvailableDisksForVmfs] Removed all candidates because all LUNs in the datastore are not visible by the same hosts or have the same transport types

      Important: VMware recommends not to turn off the filters. Disabling these filters may result in storage device corruption or performance degradation. For more information, see the Storage Filtering section of the vSphere Storage Guide

Note: Previously If the lun is force mounted  and detected as snapshot, The datastore cannot be expanded
          How to check if the lun is force mounted: 
++ login to putty session 
++ esxcfg-info -a | less
++ search for naa.xxxxxxxxx id and look for "Is Force mounted" value
++ If it's set to True, see resolution https://knowledge.broadcom.com/external/article/345232/force-mounting-a-lun-on-an-esxi-host-fai.html

Environment

VMware vSphere ESXi 7.x
VMware vSphere ESXi 8.x

Cause

The VMware vCenter Server calls a specific function to get all the available extents for that datastore. After getting these extents, the vCenter Server displays these extents as available after meeting these following criteria filters:

  • LUNS are used as datastores on that ESXI host or on any other ESXi host (with exceptions to force mounted volumes).
  • LUNS are not used as Raw Device Mapping (RDMs) on that ESXi host or any other ESXi host.
  • LUNs are seen the same by all hosts.

If any of the above filters do not match, you will not be able to increase or extend the datastore

.

Resolution

NOTE: Before starting, verify LUN presentation is the same on all hosts as this is a common, legitimate cause and should not be worked around.

To resolve this issue, follow either one of the two options:

Increase or expand the datastore directly from the ESXi host

  1. From the Web Client, right click the ESXi host and select Storage > Rescan Storage. This forces the ESXi host to sync with the changes on the underlying expanded devices.


     
  2. Once the rescan storage completes, navigate to the Datastore View and select the datastore to be expanded.
  3. Navigate to Configure > Device Backing and select Extent Name. You should see the Device Details reflecting the updated capacity from the expansion that was done from the SAN.

    Note: In this example, the datastore iscsi1 was configured for 25GB and the underlying device was expanded to 400GB.

If the underlying device does not reflect the underlying information, attempt another host level rescan so that the fields highlighted in the image above indicate the desired change. Only follow these steps if the increase in the capacity of the underlying device is detected:

  1. Log in directly to one of the ESXi host that has access to the datastore through the GUI (Host Client) using root credentials.
  2. Navigate to Storage > Datastores and select the datastore you are trying to expand or increase.
  3. Right-click on the datastore and select Increase Capacity. Alternatively, select the Increase capacity button.


    You should be prompted to select one of two options:

    Add an extent to existing VMFS datastore
    Expand an existing VMFS datastore extent

    Note: The first option is to create a multi-extent datastores by spanning (adding) a new volume to the existing datastore. The second option is to increase the capacity for the existing device itself. In this example, the second option is used:
     
  4. Select the VMFS partition as highlighted in the image and use the side-scroller to indicate the range of the expansion that would need to be done and click Next.


    Note: Sometimes, the volume expansion from the host client might fail with the following error -  'Failed to expand VMFS datastore - Cannot parse "NaN" as a long value'. 

    Login to the ssh session of the ESXi host and validate the usableSectors using the below commands.

    Run 'partedUtil getptbl /vmfs/devices/disks/naa.xxxxxxxxxxxxxxxxxxx'

    Sample output: 
    diskSize (42949672960) AlternateLBA (21474836479) LastUsableLBA (21474836446) NewLastUsableLBA (42949672926)
    gpt
    2673493 255 63 42949672960
    1 2048 21474836446 AA31E02A400F11DB9590000C2911D1B8 vmfs 0
     

    Run 'partedUtil getUsableSectors /vmfs/devices/disks/naa.xxxxxxxxxxxxxxxxxxx' to get the details of the usable sectors of this device

    Sample output:
    diskSize (42949672960) AlternateLBA (21474836479) LastUsableLBA (21474836446) NewLastUsableLBA (42949672926)
    34 42949672926

    In this case we can see that the disk size is 42949672960 but the available disk space for actual usage is 42949672926
    In such cases, when we expand the datastore using the host client we need to leave few MBs in the end instead of using the complete capacity for expansion.

  5. Click Finish. You should now be able to view the datastore expansion as completed.

  6. Immediately rescan all hosts by “rescan adapter” and “rescan vmfs” from vCenter.
    Or by commands on all hosts:
    $ esxcli storage core adapter rescan --all 
    $     vmkfstools -V

Important: 
As this was done directly on the ESXi host, VMware recommends to do a cluster level storage rescan so that all of the ESXi host accessing the same datastore are sync with the expansion that was recently completed. If a cluster level rescan is not done from the vCenter Server, there is a good chance that the vCenter Server might not readily detect the changes which may lead to disparity in the current space/usage of the datastore. VMware recommends to always do a cluster level rescan on the vCenter Server level when the datastore is increased or expanded directly from the ESXi host.

Temporarily disable the Same Host and Transport Filter

Important: VMware does not recommend to turn off or disable this filter. After temporarily turning off or disabling this filter, increase or expand the datastore. Once this is completed, ensure this filter is turned back on or re-enabled. Disabling these filters may result in storage device corruption or performance degradation that might be caused by an unsupported use of storage devices. For more information, see or more information, see the Storage Filtering section in the vSphere Storage Guide.

  1. Look for parameter "config.vpxd.filter.sameHostsAndTransportsFilter" under "Advanced vCenter Server Settings" and set it to false.
  2. Do not add the parameter if it already exists, as adding a duplicate parameter can cause vCenter instability. Instead, edit the existing parameter.

    Note: No reboot is required for changes to take into effect. Logging out and logging back in to the vCenter Server should be enough for changes to take into effect.
  3. Once completed, change "config.vpxd.filter.sameHostsAndTransportsFilter" under "Advanced vCenter Server Settings" and set it to true. For more information, see the Storage Filtering section of the vSphere Storage Guide.



Additional Information

Powered by Wolken Software