Investigating an out of resources error on a datastore

Products

• VMware
• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

This article explains the limit to the number of files allowed on a datastore and the steps required to resolve the issue.

Symptoms:

• Virtual machine fails to power on.
  
• You cannot create a virtual disk.
  
• There are more than 30000 files on the datastore. You can determine this with the find /vmfs/volumes/<data name store> -type f | wc -l </data>command.
  
• You see entries similar to these in the log file /var/log/vmkernel :
  
  vmkernel.16:Jul 2 07:57:44 srvesx89 vmkernel: 16:23:58:16.675 cpu2:25710)WARNING: World: vm 25715: 699: init fn swap failed with: No swap file!
  vmkernel.16:Jul 2 07:57:44 srvesx89 vmkernel: 16:23:58:16.741 cpu3:25712)WARNING: Swap: vm 25712: 1480: Failed to create swap file '/volumes/<volume UUID>/<swap file>.vswp': Out of resources

Environment

VMware ESX Server 3.0.x
VMware ESXi 4.0.x Installable
VMware ESXi 4.1.x Embedded
VMware VirtualCenter 2.0.x
VMware ESX 4.0.x
VMware vSphere ESXi 5.5
VMware ESXi 4.0.x Embedded
VMware VirtualCenter 2.5.x
VMware ESXi 3.5.x Embedded
VMware vCenter Server 4.1.x
VMware ESX Server 3.5.x
VMware ESXi 4.1.x Installable
VMware vCenter Server 4.0.x
VMware ESX 4.1.x
VMware ESXi 3.5.x Installable

Resolution

This issue occure because a VMFS-3 datastore can only have a maximum of 30720 files.

To resolve this issue:

• Run this command to identify how many file handles are in use on the datastore:
  
  vmkfstools –P -v 10 /vmfs/volumes/<data label store>
  
  </data>The output looks similar to:
  
  VMFS-3.21 file system spanning 1 partitions.
  File system label (if any): LUN-0
  Mode: public
  Capacity 34091302912 (32512 file blocks * 1048576), 1151336448 (1098 blocks) avail
  Volume Creation Time: Tue Apr 3 16:09:50 2007
  Files (max/free): 30720/30631
  Ptr Blocks (max/free): 61440/61409
  Sub Blocks (max/free): 3968/3906
  UUID: 4612b48e-be24f710-2df7-0013725ddc92
  Partitions spanned (on "lvm"):
  vmhba2:1:0:1
  
  The Files row describes the maximum number of files and the number of free file handles remaining.
  
  
• Look for an excessive quantity of vmware*.log files. If there there is an over abundance of generational log files (vmware-###.log and up), delete them.

Additional Information

调查数据存储上的资源不足错误