The vmkernel.log file shows message "failed to get lock on file slotsfile" on NFS datastores when Storage I/O Control (SIOC) statistics collection is enabled

Products

• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• Storage IO Control (SIOC) or Storage I/O Control statistics collection is enabled for NFS datastore(s)
• The following message is observed in the vmkernel.log relating to the file slotsfile:

YYYY-MM-DDTHH:MM:SS cpu35:34406)WARNING: NFSLock: 2210: File is being locked by a consumer on host <hostname> with exclusive lock. 
YYYY-MM-DDTHH:MM:SS cpu35:34406)NFSLock: 2966: failed to get lock on file slotsfile 0x430500b18000 on <NFS hostname> (<NFS IP>): Busy

• The following message is observed in the vmkernel.log relating to the file access

YYYY-MM-DDTHH:MM:SS cpu10:34406)WARNING: NFSLock: 2210: File is being locked by a consumer on host <hostname> with exclusive lock. 
YYYY-MM-DDTHH:MM:SS cpu10:34406)NFSLock: 2966: failed to get lock on file access 0x430500af3e00 on <NFS hostname> (<NFS IP>): Busy

Environment

VMware vSphere ESXi 6.7
VMware vSphere ESXi 6.5
VMware vSphere ESXi 6.0
VMware vSphere ESXi 5.5

Cause

The file slotsfile is used by Storage I/O Control to track performance metrics and help with SIOC decision making.  As the slotsfile is shared between all ESXi hosts accessing the NFS datastore and participating in SIOC, lock contention occurs when multiple hosts try to access the file at the same time and this message is logged.

The file access is created by each ESXi host on a Storage I/O Control enabled datastore to test that the datastore is writeable.  When multiple hosts try to create the file at the same time, this message is logged.

Resolution

These messages are expected in an environment with Storage I/O Control or Storage I/O Control statistics collection enabled for NFS datastores and can be safely ignored.

If you are not using Storage I/O Control, the messages can be prevented by disabling Storage I/O Control and Storage I/O Control statistics collection:

1. From the vSphere Web Client, navigate to Home -> Storage.
2. Select the NFS datastore and select to the Configure tab.
3. Select the Edit button next to Datastore Capabilities on the General link.
4. Deselect the Enable Storage I/O Control checkbox.
5. Select the checkbox for Disable Storage I/O statistics collection (applicable only if Storage I/O Control is disabled).
6. Click OK.
7. Perform this step for each NFS datastore in the environment.