Investigating Datastore usage

Products

• VMware vSphere ESXi

Issue/Introduction

Checking datastore usage for inconsistencies on VMFS and or NFS datastores.

Cause

Common causes for higher then expected datastore usage:

Old vm-support log bundles.
Virtual machines that are not being used and are not needed.
VMDKs  that are no longer attached to vms but are remaining on the datastore.
Old log files that are no longer needed.
ISO files that were copied to the system.

Resolution

• ls gives data on individual files based on the difference between the end-of-file (the largest offset where data is written) and the beginning-of-file, whether or not blocks were actually allocated to the file.

  A 10GB file (as reported by ls) may not have 10GB of data written to it.

• Get a list of all files/provisioned disks on the datastore:
  • ls -lahR > ls-R.txt



• Filter for the files that would typically consume the larger amount of space on the datastore.
  • egrep -i "sesparse|flat|delta" ls-R.txt
    
    
    
• To view the column of the files sizes.
  • egrep -i "sesparse|flat|delta" ls-R.txt  | awk {'print $5'}
    
    
    
• At this point you will see that the file sizes will consist of files that are (M) (G) or even (T)
• In this example we want to look only are the larger Gigabyte files.
  • egrep -i "sesparse|flat|delta" ls-R.txt  | awk {'print $5'} | grep -i G

 

• We also want to know know many files are in the list in this example we have 102 files that are in Gigabytes

egrep -i "sesparse|flat|delta" ls-R.txt  | awk {'print $5'} | grep -i G | wc -l
102

• Now we will look to prepare the file sizes to be added up.

egrep -i "sesparse|flat|delta" ls-R.txt  | awk {'print $5'} | grep -i G | head -102 | paste -sd+ - | bc
2664.28

• From the above example the files provisioned disks/snapshots total 2664 G

 

=======================

• du shows the size that is actually consumed by an individual file.
  • Specifying the -s flag reports the total blocks for all specified files or all files in a directory. The -h flag is to print the sizes in human readable format. (e.g., 1K 234M 2G)
  • The if we also add the -c flag is will produce a grand total for all the above folders/files

du -shc *

 

<Snippet of an example>
.
.

80.6G   <VM1>
65.6G   <VM2>
65.6G   <VM3>
51.9G   <VM4>
65.6G   <VM5>
139.2M  <VM6>
2.4T    total

=======================

• df shows the blocks allocated in the entire file system, including inodes and other meta data.

df -h 

 

Additional Information

This will look for files that are 3 directories down from the current location as well as files that are above 7 M

 find . -maxdepth 3 -type f -size +7M -exec ls -lah {} + | sort -rh

 

If you are unable to determine what is consuming the disk space, use the find command to locate all files matching a given criteria.
For example, to find files within / that are larger than 10MB without traversing mount points, use the command:
find / -size +10M -exec du -h {} \; | less

 

To find files within /var/ that are larger than 1MB without traversing mount points, use the command:
 
find /var/ -size +1M -mount -exec du -h {} \; | less