The VMFS datastore could not be expanded Error: Failed to expand VMFS datastore datastore1 - Cannot change the host configuration

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms : 

• Unable to expand the datastore size from vSphere client.
  
  Error: Cannot change the host configuration.
  
   
  
  
• Unable to expand the datastore size from Host Client :
   
  Error: Failed to expand VMFS datastore datastore1 - Cannot change the host configuration.

         

• Backend LUN was expanded but unable to expand the datastore. 

 

Environment

VMware vSphere ESXi

Cause

• Underlying LUN for the datastore is identified as a snapshot. Lun detected as snapshot loses features like LUN size expansion and auto-mount volume to ESXi host, until the volume is resignatured or snapshot scenario is fixed.
  
  
• In "/var/run/log/vmkernel.log" , you will see below events: 

# Example 1:
LVM: 8445: Device naa.####################:1 detected to be a snapshot:
LVM: 8452: queried disk ID: <type 1, len 17, lun 36, devType 0, scsi 0, h(id) 7683208289187576905>
LVM: 8459: on-disk disk ID: <type 1, len 17, lun 17, devType 0, scsi 0, h(id) 7683208289187576905>

#Example 2:
LVM: 11764: Device naa.####################:1 detected to be a snapshot:
LVM: 11770:   queried disk ID: <type 2, len 22, lun 0, devType 0, scsi 0, h(id) 15436777671286394200>
LVM: 11777:   on-disk disk ID: <type 2, len 22, lun 0, devType 0, scsi 0, h(id) 16841058529903986994>

Note:

• • In Example 1, LUN was originally presented to a host as LUN 17. It was then unmapped and remapped (or presented to a different cluster) using LUN 36.
  • In Example 2, The "h(id)" (Hardware ID/Hash) has changed completely. This usually happens during Storage Replication or a Storage Array Migration. The data was copied from one physical serial number/WWN to a completely different one.

 

Resolution

If you are in the situation of Example 1, you can fix the problem without touching ESXi.

• Storage vmotion the VMs from the datastore backed by snapshot LUN.

• Unmount the "Snapshot" LUN from ESXi.

• On the Storage Array, change the Host-LUN ID mapping to match the original (e.g., change it from 36 back to 17).

• Rescan the ESXi host.

Follow the below steps to resolve the issue for both the examples - 

• Storage vmotion the VMs running on the force mounted snapshot detected volumes. 
  
  Refer - https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere/8-0/migrate-a-virtual-machine-with-storage-vmotion.html
  
  
• Unmount the datastore from the host.
  
  Refer - https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/vsphere/7-0/vsphere-storage/working-with-datastores-in-vsphere-storage-environment/basic-administrative-operations-for-vsphere-datastores/unmount-datastores.html
  
  Command : 
  $ esxcli storage filesystem unmount --volume-label=<label>, -l=<label> | --volume-uuid=<VMFS-UUID>, -u=<VMFS-UUID>
  
  
• Perform a resignature of the datastore to generate a new UUID and remove the snapshot flag.
  
  Command :
  $ esxcli storage vmfs snapshot resignature -l "datastore-name"
  
  

• Remount the datastore post-resignature.
  
  Refer -How to List, mount and unmount a volume/VMFS Datastore.
  
  

• Successfully expand the datastore to include the newly added LUN capacity.

  Refer - https://knowledge.broadcom.com/external/article/314373/extending-or-increasing-a-datastore-thro.html

Additional Information

Troubleshooting LUNs detected as snapshot LUNs in vSphere

Failed to expand VMFS datastore - Cannot change the host configuration