Maintaining write-ordering and write-through integrity using NFS in an ESX/ESXi environment

Products

• VMware vSphere ESXi

Issue/Introduction

Certain applications such as databases, require write-order/write-through integrity from the underlying storage. This article provides information about NFS datastore deployment in VMware ESX/ESXi, and details about the support for write-ordering and write-through integrity for such applications.

Environment

VMware ESXi 4.1.x Embedded
VMware vSphere ESXi 5.5
VMware ESX 4.0.x
VMware ESX Server 3.0.x
VMware vSphere ESXi 5.0
VMware vSphere ESXi 5.1
VMware ESX Server 3.5.x
VMware ESXi 4.1.x Installable
VMware ESXi 3.5.x Installable
VMware ESXi 3.5.x Embedded
VMware ESXi 4.0.x Installable
VMware ESX 4.1.x
VMware ESXi 4.0.x Embedded

Resolution

The write-ordering and write-through integrity for NFS storage are not satisfied with NFS in an VMware ESX/ESXi environment.

An NFS datastore, when mounted on an ESX/ESXi host, IO from guest goes via virtual SCSI emulation to NFS datastore. A Virtual Machine Disk (VMDK) file on an NFS datastore appears as a SCSI disk within the virtual machine’s guest operating system, all the SCSI operations on this datastore by Guest Operating System are translated by the SCSI emulation layer to the appropriate NFS protocol operations.

The database applications running in the Guest VM may require write-ordering and write-through integrity on the NFS datastore backing this VMDK file. These database applications require the following SCSI level write-ordering and write-through integrity on this VMDK file. 

• Forced Unit Access (FUA)/Write-Through

• Torn I/O Protection

Forced Unit Access (FUA) and Write-Through command is implemented by issuing a WRITE SCSI command to the disk with the FUA bit set. In the ESX/ESXi environment this SCSI command is originated in the guest OS and reaches the SCSI emulation layer. Currently this FUA bit is NOT honored by the SCSI emulation layer, hence Write-Through is NOT guaranteed on NFS datastores.  

Torn I/O Protection. The SCSI specification says that a solution must provide sector alignment and sizing in a way that prevents torn I/O including splitting I/Os across various I/O entities in the I/O path. In other words, storage must ensure that all of the data for a transaction is written and never reports success when partial writes occur. This requirement must be satisfied by both the NFS storage provider as well as the NFS client on the ESX/ESXi. The NFS client supports this requirement by reporting success only when all the split I/Os are written successfully, any partial writes are reported as errors.