Utility to list the NFS v3 lock files and their contents in human readable format.
search cancel

Utility to list the NFS v3 lock files and their contents in human readable format.

book

Article ID: 311914

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

The purpose of this article is to help you with the usage of the script that provides details about the .lck file in user-readable format.


Symptoms:

When one connects an ESX host to a supported NFSv3 server, ESX does not use the traditional Network Lock Manager (NLM) protocol to handle NFS locking. Instead, we create a .lck file for each associated file and populate said file with the hostname in plain text, and also indirectly points to the inode of the file on the NFS export.


Environment

VMware vSphere ESXi 8.0.x

Cause

There can be times when you are unsure which lock file belongs to which file. A good example of this would be a shared swap directory which may contain multiple swap files from multiple VMs.

Resolution

A script has been added to quickly obtain the .lck file details that are part of NFSv3 datastore(s). The article explains about the different use cases of the script.

Use case 1 - List details of all .lck files part of a VM

1. SSH into ESXi host.
2. Find out the appropriate VM directory.
3. For example, the VM is located at /vmfs/volumes/cfcc81c4-e203c1b2/Ubuntu-test/ 
4. The VM directory contains the following files.

[root@esx-host:~] ls /vmfs/volumes/cfcc81c4-e203c1b2/Ubuntu-test/ -lai
total 364
100751808 drwxr-xr-x    3 root     root          4096 Dec 28 10:13 .
67157059 drwxr-xr-x    3 root     root            25 Dec 28 10:13 ..
100772701 -rwxrwxr-x    1 root     root            92 Dec 28 10:28 .lck-5cab010600000000
100772705 -rwxrwxr-x    1 root     root            92 Dec 28 10:28 .lck-5eab010600000000
100772704 -rwxrwxr-x    1 root     root            92 Dec 28 10:28 .lck-cc59010600000000
100751871 -rwxrwxr-x    1 root     root            92 Dec 28 10:28 .lck-fe59010600000000
100772702 -rw-------    1 root     root      33554432 Dec 28 10:13 Ubuntu-test-4c650d76.vswp
100751820 -rw-------    1 root     root       1048576 Dec 28 10:13 Ubuntu-test-flat.vmdk
100772711 -rw-------    1 root     root          8684 Dec 28 10:13 Ubuntu-test.nvram
100752055 -rw-r--r--    1 root     root          7386 Dec 28 10:13 Ubuntu-test.scoreboard
100751860 -rw-------    1 root     root           443 Dec 28 10:13 Ubuntu-test.vmdk
100751862 -rw-r--r--    1 root     root             0 Dec 28 10:13 Ubuntu-test.vmsd
100772712 -rwxr-xr-x    1 root     root          1776 Dec 28 10:13 Ubuntu-test.vmx
100772700 -rw-------    1 root     root             0 Dec 28 10:13 Ubuntu-test.vmx.lck
   415014 drwxr-xr-x    2 root     root            94 Dec 28 10:13 stats
100772703 -rw-r--r--    1 root     root        324993 Dec 28 10:28 vmware.log
100751870 -rw-------    1 root     root      83886080 Dec 28 10:13 vmx-Ubuntu-test-0be6d08851c29d9cabc7ef29fdb1eb42f6924a6cdd1068719fc49b90d29e0516-1.vswp

5. To list details of all the four .lck  files above, the following command can be issued. Please note that the script only takes absolute file path with all links resolved, as input.

[root@esx-host:~] python /usr/lib/vmware/vm-support/bin/nfsLockInfo.pyc /vmfs/volumes/cfcc81c4-e203c1b2/Ubuntu-test/
VM Dir     : /vmfs/volumes/cfcc81c4-e203c1b2/Ubuntu-test/
Local Host : localhost
 
Lock number: 1
Lock File  : .lck-fe59010600000000
Inode      : 0x60159fe (100751870)
Locked file: vmx-Ubuntu-test-0be6d08851c29d9cabc7ef29fdb1eb42f6924a6cdd1068719fc49b90d29e0516-1.vswp
Counter    : 37
Version    : 0xabcd0123
LockType   : 1 (EXCLUSIVE)
Host Name  : localhost
Host UUID  : 658a91df-5ea9-35d7-40b7-02012bfd98d0
Non-Shared : 0xff
Lock Dump  :
===============================================
0000000 2500 0000 0000 0000 2301 cdab ffff ffff
0000010 0100 0000 6c6f 6361 6c68 6f73 7400 0000
0000020 0000 0000 0000 0000 0000 0000 0000 0000
0000030 0000 0000 658a 91df 5ea9 35d7 40b7 0201
0000040 2bfd 98d0 0000 0000 0000 0000 0000 0000
0000050 0000 00ff 0000 0000 0000 0000
===============================================
 
Lock number: 2
Lock File  : .lck-5cab010600000000
Inode      : 0x601ab5c (100772700)
Locked file: Ubuntu-test.vmx.lck
Counter    : 36
Version    : 0xabcd0123
LockType   : 1 (EXCLUSIVE)
Host Name  : localhost
Host UUID  : 658a91df-5ea9-35d7-40b7-02012bfd98d0
Non-Shared : 0xff
Lock Dump  :
===============================================
0000000 2400 0000 0000 0000 2301 cdab ffff ffff
0000010 0100 0000 6c6f 6361 6c68 6f73 7400 0000
0000020 0000 0000 0000 0000 0000 0000 0000 0000
0000030 0000 0000 658a 91df 5ea9 35d7 40b7 0201
0000040 2bfd 98d0 0000 0000 0000 0000 0000 0000
0000050 0000 00ff 0000 0000 0000 0000
===============================================
 
Lock number: 3
Lock File  : .lck-5eab010600000000
Inode      : 0x601ab5e (100772702)
Locked file: Ubuntu-test-4c650d76.vswp
Counter    : 35
Version    : 0xabcd0123
LockType   : 1 (EXCLUSIVE)
Host Name  : localhost
Host UUID  : 658a91df-5ea9-35d7-40b7-02012bfd98d0
Non-Shared : 0xff
Lock Dump  :
===============================================
0000000 2300 0000 0000 0000 2301 cdab ffff ffff
0000010 0100 0000 6c6f 6361 6c68 6f73 7400 0000
0000020 0000 0000 0000 0000 0000 0000 0000 0000
0000030 0000 0000 658a 91df 5ea9 35d7 40b7 0201
0000040 2bfd 98d0 0000 0000 0000 0000 0000 0000
0000050 0000 00ff 0000 0000 0000 0000
===============================================
 
Lock number: 4
Lock File  : .lck-cc59010600000000
Inode      : 0x60159cc (100751820)
Locked file: Ubuntu-test-flat.vmdk
Counter    : 34
Version    : 0xabcd0123
LockType   : 1 (EXCLUSIVE)
Host Name  : localhost
Host UUID  : 658a91df-5ea9-35d7-40b7-02012bfd98d0
Non-Shared : 0xff
Lock Dump  :
===============================================
0000000 2200 0000 0000 0000 2301 cdab ffff ffff
0000010 0100 0000 6c6f 6361 6c68 6f73 7400 0000
0000020 0000 0000 0000 0000 0000 0000 0000 0000
0000030 0000 0000 658a 91df 5ea9 35d7 40b7 0201
0000040 2bfd 98d0 0000 0000 0000 0000 0000 0000
0000050 0000 00ff 0000 0000 0000 0000
===============================================
6. The script outputs all fields of the .lck file in user readable format, along with hexdump of the contents for verification (if needed). 

Use case 2 - Find the .lck file of a given parent file

1. SSH into ESXi host
2. Find out the appropriate VM file
3. For example, the VM file is /vmfs/volumes/cfcc81c4-e203c1b2/Ubuntu-test/Ubuntu-test-flat.vmdk 
4. Use the following command to get the .lck  file corresponding to Ubuntu-test-flat.vmdk file. Please note that the script only takes absolute file path with all links resolved, as input.

[root@esx-host:~] python /usr/lib/vmware/vm-support/bin/nfsLockInfo.pyc /vmfs/volumes/cfcc81c4-e203c1b2/Ubuntu-test/Ubuntu-test-flat.vmdk
VM Dir     : /vmfs/volumes/cfcc81c4-e203c1b2/Ubuntu-test
Local Host : localhost
 
Lock number: 1
Lock File  : .lck-cc59010600000000
Inode      : 0x60159cc (100751820)
Locked file: Ubuntu-test-flat.vmdk
Counter    : 50
Version    : 0xabcd0123
LockType   : 1 (EXCLUSIVE)
Host Name  : localhost
Host UUID  : 658a91df-5ea9-35d7-40b7-02012bfd98d0
Non-Shared : 0xff
Lock Dump  :
===============================================
0000000 3200 0000 0000 0000 2301 cdab ffff ffff
0000010 0100 0000 6c6f 6361 6c68 6f73 7400 0000
0000020 0000 0000 0000 0000 0000 0000 0000 0000
0000030 0000 0000 658a 91df 5ea9 35d7 40b7 0201
0000040 2bfd 98d0 0000 0000 0000 0000 0000 0000
0000050 0000 00ff 0000 0000 0000 0000
===============================================


[root@esx-host:~] python /usr/lib/vmware/vm-support/bin/nfsLockInfo.pyc /vmfs/volumes/cfcc81c4-e203c1b2/Ubuntu-test/Ubuntu-test-flat.vmdk
VM Dir     : /vmfs/volumes/cfcc81c4-e203c1b2/Ubuntu-test
Local Host : localhost
 
Lock number: 1
Lock File  : .lck-cc59010600000000
Inode      : 0x60159cc (100751820)
Locked file: Ubuntu-test-flat.vmdk
Counter    : 50
Version    : 0xabcd0123
LockType   : 1 (EXCLUSIVE)
Host Name  : localhost
Host UUID  : 658a91df-5ea9-35d7-40b7-02012bfd98d0
Non-Shared : 0xff
Lock Dump  :
===============================================
0000000 3200 0000 0000 0000 2301 cdab ffff ffff
0000010 0100 0000 6c6f 6361 6c68 6f73 7400 0000
0000020 0000 0000 0000 0000 0000 0000 0000 0000
0000030 0000 0000 658a 91df 5ea9 35d7 40b7 0201
0000040 2bfd 98d0 0000 0000 0000 0000 0000 0000
0000050 0000 00ff 0000 0000 0000 0000
===============================================


Additional Information

Impact/Risks:

Manually deleting .lck files should never be done because it can cause random data corruption which may not be immediately visible, or, under the express instruction of a VMware Technical Support Engineer.


Powered by Wolken Software