Utility to list the NFS v3 lock files and their contents in human readable format.
Article ID: 311914
Updated On:
Products
Issue/Introduction
The purpose of this article is to help you with the usage of the script that provides details about the .lck file in user-readable format.
Symptoms:
When one connects an ESX host to a supported NFSv3 server, ESX does not use the traditional Network Lock Manager (NLM) protocol to handle NFS locking. Instead, we create a .lck file for each associated file and populate said file with the hostname in plain text, and also indirectly points to the inode of the file on the NFS export.
Environment
Cause
There can be times when you are unsure which lock file belongs to which file. A good example of this would be a shared swap directory which may contain multiple swap files from multiple VMs.
Resolution
A script has been added to quickly obtain the .lck file details that are part of NFSv3 datastore(s). The article explains about the different use cases of the script.
Use case 1 - List details of all .lck files part of a VM
1. SSH into ESXi host.
2. Find out the appropriate VM directory.
3. For example, the VM is located at /vmfs/volumes/cfcc81c4-########/Ubuntu-test/
4. The VM directory contains the following files.
[root@esx-host:~] ls /vmfs/volumes/cfcc81c4-########/Ubuntu-test/ -lai
total 364
100751808 drwxr-xr-x 3 root root 4096 Dec 28 10:13 .
67157059 drwxr-xr-x 3 root root 25 Dec 28 10:13 ..
100772701 -rwxrwxr-x 1 root root 92 Dec 28 10:28 .lck-5cab010600000000
100772705 -rwxrwxr-x 1 root root 92 Dec 28 10:28 .lck-5eab010600000000
100772704 -rwxrwxr-x 1 root root 92 Dec 28 10:28 .lck-cc59010600000000
100751871 -rwxrwxr-x 1 root root 92 Dec 28 10:28 .lck-fe59010600000000
100772702 -rw------- 1 root root 33554432 Dec 28 10:13 Ubuntu-test-4c650d76.vswp
100751820 -rw------- 1 root root 1048576 Dec 28 10:13 Ubuntu-test-flat.vmdk
100772711 -rw------- 1 root root 8684 Dec 28 10:13 Ubuntu-test.nvram
100752055 -rw-r--r-- 1 root root 7386 Dec 28 10:13 Ubuntu-test.scoreboard
100751860 -rw------- 1 root root 443 Dec 28 10:13 Ubuntu-test.vmdk
100751862 -rw-r--r-- 1 root root 0 Dec 28 10:13 Ubuntu-test.vmsd
100772712 -rwxr-xr-x 1 root root 1776 Dec 28 10:13 Ubuntu-test.vmx
100772700 -rw------- 1 root root 0 Dec 28 10:13 Ubuntu-test.vmx.lck
415014 drwxr-xr-x 2 root root 94 Dec 28 10:13 stats
100772703 -rw-r--r-- 1 root root 324993 Dec 28 10:28 vmware.log
100751870 -rw------- 1 root root 83886080 Dec 28 10:13 vmx-Ubuntu-test-0be6d08851c29d9cabc7ef29fdb1eb42f6924a6cdd1068719fc49b90d29e0516-1.vswp
5. To list details of all the four .lck files above, the following command can be issued. Please note that the script only takes absolute file path with all links resolved, as input.
[root@esx-host:~] python /usr/lib/vmware/vm-support/bin/nfsLockInfo.pyc /vmfs/volumes/cfcc81c4-e203c1b2/Ubuntu-test/
VM Dir : /vmfs/volumes/cfcc81c4-e203c1b2/Ubuntu-test/
Local Host : localhost
Lock number: 1
Lock File : .lck-fe59010600000000
Inode : 0x60159fe (100751870)
Locked file: vmx-Ubuntu-test-0be6d08851c29d9cabc7ef29fdb1eb42f6924a6cdd1068719fc49b90d29e0516-1.vswp
Counter : 37
Version : 0xabcd0123
LockType : 1 (EXCLUSIVE)
Host Name : localhost
Host UUID : ########-####-####-####-########98d0
Non-Shared : 0xff
Lock Dump :
===============================================
0000000 2500 0000 0000 0000 2301 cdab ffff ffff
0000010 0100 0000 6c6f 6361 6c68 6f73 7400 0000
0000020 0000 0000 0000 0000 0000 0000 0000 0000
0000030 0000 0000 658a 91df 5ea9 35d7 40b7 0201
0000040 2bfd 98d0 0000 0000 0000 0000 0000 0000
0000050 0000 00ff 0000 0000 0000 0000
===============================================
Lock number: 2
Lock File : .lck-5cab010600000000
Inode : 0x601ab5c (100772700)
Locked file: Ubuntu-test.vmx.lck
Counter : 36
Version : 0xabcd0123
LockType : 1 (EXCLUSIVE)
Host Name : localhost
Host UUID : ########-####-####-####-########98d0
Non-Shared : 0xff
Lock Dump :
===============================================
0000000 2400 0000 0000 0000 2301 cdab ffff ffff
0000010 0100 0000 6c6f 6361 6c68 6f73 7400 0000
0000020 0000 0000 0000 0000 0000 0000 0000 0000
0000030 0000 0000 658a 91df 5ea9 35d7 40b7 0201
0000040 2bfd 98d0 0000 0000 0000 0000 0000 0000
0000050 0000 00ff 0000 0000 0000 0000
===============================================
Lock number: 3
Lock File : .lck-5eab010600000000
Inode : 0x601ab5e (100772702)
Locked file: Ubuntu-test-4c650d76.vswp
Counter : 35
Version : 0xabcd0123
LockType : 1 (EXCLUSIVE)
Host Name : localhost
Host UUID : ########-####-####-####-########98d0
Non-Shared : 0xff
Lock Dump :
===============================================
0000000 2300 0000 0000 0000 2301 cdab ffff ffff
0000010 0100 0000 6c6f 6361 6c68 6f73 7400 0000
0000020 0000 0000 0000 0000 0000 0000 0000 0000
0000030 0000 0000 658a 91df 5ea9 35d7 40b7 0201
0000040 2bfd 98d0 0000 0000 0000 0000 0000 0000
0000050 0000 00ff 0000 0000 0000 0000
===============================================
Lock number: 4
Lock File : .lck-cc59010600000000
Inode : 0x60159cc (100751820)
Locked file: Ubuntu-test-flat.vmdk
Counter : 34
Version : 0xabcd0123
LockType : 1 (EXCLUSIVE)
Host Name : localhost
Host UUID : ########-####-####-####-########98d0
Non-Shared : 0xff
Lock Dump :
===============================================
0000000 2200 0000 0000 0000 2301 cdab ffff ffff
0000010 0100 0000 6c6f 6361 6c68 6f73 7400 0000
0000020 0000 0000 0000 0000 0000 0000 0000 0000
0000030 0000 0000 658a 91df 5ea9 35d7 40b7 0201
0000040 2bfd 98d0 0000 0000 0000 0000 0000 0000
0000050 0000 00ff 0000 0000 0000 0000
===============================================
6. The script outputs all fields of the .lck file in user readable format, along with hexdump of the contents for verification (if needed).
Use case 2 - Find the .lck file of a given parent file
1. SSH into ESXi host
2. Find out the appropriate VM file
3. For example, the VM file is /vmfs/volumes/cfcc81c4-########/Ubuntu-test/Ubuntu-test-flat.vmdk
4. Use the following command to get the .lck file corresponding to Ubuntu-test-flat.vmdk file. Please note that the script only takes absolute file path with all links resolved, as input.
[root@esx-host:~] python /usr/lib/vmware/vm-support/bin/nfsLockInfo.pyc /vmfs/volumes/cfcc81c4-e203c1b2/Ubuntu-test/Ubuntu-test-flat.vmdk
VM Dir : /vmfs/volumes/cfcc81c4-########/Ubuntu-test
Local Host : localhost
Lock number: 1
Lock File : .lck-cc59010600000000
Inode : 0x60159cc (100751820)
Locked file: Ubuntu-test-flat.vmdk
Counter : 50
Version : 0xabcd0123
LockType : 1 (EXCLUSIVE)
Host Name : localhost
Host UUID : ########-####-####-####-########98d0
Non-Shared : 0xff
Lock Dump :
===============================================
0000000 3200 0000 0000 0000 2301 cdab ffff ffff
0000010 0100 0000 6c6f 6361 6c68 6f73 7400 0000
0000020 0000 0000 0000 0000 0000 0000 0000 0000
0000030 0000 0000 658a 91df 5ea9 35d7 40b7 0201
0000040 2bfd 98d0 0000 0000 0000 0000 0000 0000
0000050 0000 00ff 0000 0000 0000 0000
===============================================
[root@esx-host:~] python /usr/lib/vmware/vm-support/bin/nfsLockInfo.pyc /vmfs/volumes/cfcc81c4-e203c1b2/Ubuntu-test/Ubuntu-test-flat.vmdk
VM Dir : /vmfs/volumes/cfcc81c4-########/Ubuntu-test
Local Host : localhost
Lock number: 1
Lock File : .lck-cc59010600000000
Inode : 0x60159cc (100751820)
Locked file: Ubuntu-test-flat.vmdk
Counter : 50
Version : 0xabcd0123
LockType : 1 (EXCLUSIVE)
Host Name : localhost
Host UUID : ########-####-####-####-########98d0
Non-Shared : 0xff
Lock Dump :
===============================================
0000000 3200 0000 0000 0000 2301 cdab ffff ffff
0000010 0100 0000 6c6f 6361 6c68 6f73 7400 0000
0000020 0000 0000 0000 0000 0000 0000 0000 0000
0000030 0000 0000 658a 91df 5ea9 35d7 40b7 0201
0000040 2bfd 98d0 0000 0000 0000 0000 0000 0000
0000050 0000 00ff 0000 0000 0000 0000
===============================================
Additional Information
Manually deleting .lck files should never be done because it can cause random data corruption which may not be immediately visible, or, under the express instruction of a VMware Technical Support Engineer.
Feedback
