VDDK fails with error 14009 while backup application tries to read/write after session is disconnected

Products

• VMware vSphere ESXi

Issue/Introduction

When a backup application tries to do a read/write operations and there is already nbd failure reported by vddk, the following messages within VDDK logs appear:

MM/DD/YYYY HH:MM:SS.### : VixDiskLib: Detected DiskLib error 2338 (NBD_ERR_NETWORK_CONNECT).
MM/DD/YYYY HH:MM:SS.### : VixDiskLib: VixDiskLib_Read: Read 8192 sectors at 1493057536 failed. Error 14009 (The server refused connection) (DiskLib error 2338: NBD_ERR_NETWORK_CONNECT) at 5814.
..
..
..
MM/DD/YYYY HH:MM:SS.### : [NFC ERROR] NFC_NETWORK_ERROR: The session is in a faulted state: NfcAio_AIO
MM/DD/YYYY HH:MM:SS.### : [NFC ERROR] NFC_NETWORK_ERROR: The session is in a faulted state: NfcAio_Wait
MM/DD/YYYY HH:MM:SS.### : DISKLIB-LIB : no answer. numBytes didn't reach 0. Dumping iov.
MM/DD/YYYY HH:MM:SS.### : ###### dumping content of iov ######
MM/DD/YYYY HH:MM:SS.### : READ
MM/DD/YYYY HH:MM:SS.### : startSector = 34209792
MM/DD/YYYY HH:MM:SS.### : numSectors = 8192
MM/DD/YYYY HH:MM:SS.### : numBytes = 4194304
MM/DD/YYYY HH:MM:SS.### : numEntries = 1
MM/DD/YYYY HH:MM:SS.### : entries[0] = D6392EE000 / 4194304
MM/DD/YYYY HH:MM:SS.### : PANIC: VERIFY d:/build/ob/bora-9272949/bora/lib/disklib/diskLib.c:4086 bugNr=24092

Cause

This issue occurs because in VDDK 6.7 and its first two update releases, VDDK may crash due to I/O retry after an NBD asynchronous I/O (AIO) error. If the backup application ignore the errors and continues reading or writing, it then results in an NBD network-connect error and subsequent program crash.

Resolution

This is a known issue affecting VMware VDDK 6.7.x and later versions.

• VDDK 6.7.0 Build 8173251 
• VDDK 6.7.1 Build 10362358 
• VDDK 6.7.2 Build 13015725

This issue is fixed in VMware VDDK 6.7 Update 3.