Virtual Machine Console Showing – "WRITE SAME failed. Manually zeroing"

Products

• VMware vSAN

Issue/Introduction

After restarting certain storage VMs, the VM console failed to load, and ESXi logs displayed the following message:

dm-0: WRITE SAME failed. Manually zeroing.

This occurs when ESXi attempts the WRITE SAME command for zeroing blocks, but the underlying NFS datastore does not support this operation.

Validation

• Investigation confirmed that the affected NFS datastores do not support hardware acceleration (VAAI primitives such as WRITE SAME). Verification can be performed with the following command: esxcli storage nfs list

Volume Name                Host             Share                       Vmknic   Accessible       Mounted         Connections        Read-Only        isPE   Hardware Acceleration
-------------------        ----------       ------------------------    ------   ----------       -------         -----------        ---------        -----  ---------------------
NFS1                       10.##.#.##       /NFS1                       None      true             true               1              false            false  Not Supported

• NFS latency spikes were observed:

####-0#-0#T02:56:44.819Z Wa(180) vmkwarning: cpu64:2098952)WARNING: NFS: 5965: NFS volume NFS1 performance has deteriorated. I/O latency increased from average value of 0(us) to 10157(us). Exceeded threshold 10000(us)
####-0#-0#T03:02:56.605Z Wa(180) vmkwarning: cpu110:2098952)WARNING: NFS: 5965: NFS volume NFS1 performance has deteriorated. I/O latency increased from average value of 0(us) to 10027(us). Exceeded threshold 10000(us)
####-0#-0#T03:19:39.851Z In(182) vmkernel: cpu77:2098952)NFS: 5886: NFS volume NFS1 performance has improved. I/O latency reduced from 10016(us) to 4943(us).
####-0#-0#T03:32:46.773Z In(182) vmkernel: cpu68:2098952)NFS: 5886: NFS volume NFS1 performance has improved. I/O latency reduced from 10374(us) to 4975(us).

Environment

VMware vSAN 7.x

VMware vSAN 8.x

Cause

A critical warning has been issued because a physical storage device incorrectly reported support for the WRITE_SAME VAAI primitive, which is intended to offload zeroing operations to the hardware; however, subsequent attempts to use the command failed due to an inherent device error or misconfiguration. Although NFS datastores lacking VAAI support cannot process this command, the message itself is purely informational and confirms there is no data corruption or VM failure.

Resolution

• Monitor VM performance.
• If boot times or application responsiveness are slower than expected, the message may indicate NFS storage performance limitations.
• Engage the NFS vendor to investigate NFS latency or hardware acceleration support.

Additional Information

Frequently Asked Questions for vStorage APIs for Array Integration

Best Practices:

• For large-scale deployments or performance-sensitive workloads, consider using datastores that support VAAI hardware acceleration.