md5sum result different for certain files on VMs

Products

• VMware vSphere ESXi

Issue/Introduction

Issue started after controller related activity was performed on the Storage.

 

Environment

8.0 U3

Cause

I/O errors got logged for multiple datastores in vmkernel.log.

• ScsiDeviceIO: 13055: READ CAPACITY on device "naa.xxxxxxxx" from Plugin "NMP" failed. I/O error.
 vmkwarning: cpu257:2103667)WARNING: ScsiDeviceIO: 13055: READ CAPACITY on device "naa.xxxxxxxxxx" from Plugin "NMP" failed. I/O error
• "RC Lock not free for type 1, return TXN FULL","Failed to unmap file blocks 0/4:Transaction ran out of lock space or log space" errors in vmkernel.log.
• vmkernel: cpu251:8115846)Res6: 2944: 'eamc_ps_iscsi_107': RC Lock not free for type 1, return TXN FULL
• vmkernel: cpu251:8115846)Fil6: 3810: eamc_ps_iscsi_107: <FD c19 r8> - Failed to unmap file blocks 0/4:Transaction ran out of lock space or log space

During the storage outage, the UNMAP command failed to complete its execution on the backing LUN, resulting in stale metadata locks. Because the UNMAP process requires exclusive locks to safely perform block reclamation, these orphaned locks are now preventing subsequent space reclamation tasks and causing transaction errors.

"Failed to acquire port non-exclusive lock" errors in vmkernel.log.

• vmkernel: cpu426:7993620)NetPort: 708: Failed to acquire port non-exclusive lock 0x4000034[Failure].
• vmkernel: cpu278:7994898)NetPort: 708: Failed to acquire port non-exclusive lock 0x4000037[Failure].

This error signifies that processes are failing to acquire necessary kernel locks to manage networking resources (virtual switch ports, etc.). This could be due to an issue with NIC driver / firmware.

Resolution

Storage vendor resolved the issue on Storage. NIC driver and firmware versions were upgraded to the latest versions.

• In addition, as this storage was connected via iSCSI, NIC driver and firmware were updated to the latest available recommended levels.