Error: “msg.fileio.generic” while unable to take snapshots

Products

• VMware vSphere ESXi

Issue/Introduction

• Unable to take snapshots manually and through backup
• Taking snapshot with or without quiesce or memory state will fail with the following error:
  
  An error occurred while taking a snapshot: msg.fileio.generic.
An error occurred while saving the snapshot: msg.fileio.generic.
   
• In the /vmfs/volumes/datastore/vm-name/vmware.log file, there are similar entries to:

| vmx| I120: VigorTransportProcessClientPayload: opID=254F67FE-00000087-ce-3b-ab7a seq=6800308: Receiving Snapshot.Take request.

| vmx| I120: SnapshotVMX_TakeSnapshot start: 'test', deviceState=0, lazy=0, logging=0, quiesced=0, forceNative=0, tryNative=0, saveAllocMaps=0 cb=2C30A630, cbData=325CA180

| vmx| I120: DISKLIB-LIB_CREATE : DiskLibCreateCreateParam: vmfsSparse grain size is set to 1 for '/vmfs/volumes/557050be-########-####-##########3f/VMNAME/VMNAME-000001.vmdk'

| vmx| I120: SNAPSHOT: SnapshotPrepareTakeDoneCB: Prepare phase complete (The operation completed successfully).

| vcpu-0| I120: Destroying virtual dev for scsi0:0 vscsi=43140

vcpu-0| I120: VMMon_VSCSIStopVports: No such target on adapter

vcpu-0| I120: SnapshotVMXTakeSnapshotWork: Transition to mode 0.

vcpu-0| I120: Closing all the disks of the VM.

vcpu-0| I120: Closing disk scsi0:0

vcpu-0| I120: DISKLIB-CBT : Shutting down change tracking for untracked fid 532940586.

| vcpu-0| I120: DISKLIB-CBT : Successfully disconnected CBT node.

| vcpu-0| I120: DISKLIB-VMFS : "/vmfs/volumes/557050be-########-####-##########3f/VMNAME/VMNAME-flat.vmdk" : closed.

vcpu-0| I120: SNAPSHOT: SnapshotConfigInfoReadEx: Creating new snapshot dictionary, '/vmfs/volumes/557050be-########-####-##########3f/VMNAME/VMNAME.vmsd.usd'.

| vcpu-0| I120: DISKLIB-VMFS : "/vmfs/volumes/557050be-########-####-##########3f/VMNAME/VMNAME-flat.vmdk" : open successful (65557) size = 64424509440, hd = 0. Type 3

| vcpu-0| I120: DISKLIB-LIB_BLOCKTRACK : Resuming change tracking.

vcpu-0| I120: DISKLIB-VMFS : "/vmfs/volumes/557050be-########-####-##########3f/VMNAME/VMNAME-flat.vmdk" : closed.

| vcpu-0| W110: File_GetFreeSpace: Couldn't statfs /vmfs/volumes/557050be-########-####-##########3f/VMNAME

| vcpu-0| I120: SNAPSHOT: SnapshotCheckSpaceAndSnapshotNum: Error getting the free space on the storage disk

vcpu-0| I120: Checkpoint_Unstun: vm stopped for 1187674 us

| vcpu-0| I120: SCSI: switching scsi0 to push completion mode

| vcpu-0| A115: ConfigDB: Setting scsi0:0.redo = ""

| vcpu-0| I120: DISK: OPEN scsi0:0 '/vmfs/volumes/557050be-########-####-##########3f/VMNAME/VMNAME.vmdk' persistent R[]

vcpu-0| I120: DISKLIB-VMFS : "/vmfs/volumes/557050be-########-####-##########3f/VMNAME/VMNAME-flat.vmdk" : open successful (10) size = 64424509440, hd = 571017004. Type 3

vcpu-0| I120: DISKLIB-DSCPTR: Opened [0]: "VMNAME-flat.vmdk" (0xa)

vcpu-0| I120: DISKLIB-LINK : Opened '/vmfs/volumes/557050be-########-####-##########3f/VMNAME/VMNAME.vmdk' (0xa): vmfs, 125829120 sectors / 60 GB.

| vcpu-0| I120: DISKLIB-LIB_BLOCKTRACK : Resuming change tracking.

| vcpu-0| I120: DISKLIB-CBT : Initializing ESX kernel change tracking for fid 571017004.

| vcpu-0| I120: DISKLIB-CBT : Successfuly created cbt node 2209072c-cbt.

| vcpu-0| I120: DISKLIB-CBT : Opening cbt node /vmfs/devices/cbt/2209072c-cbt

vcpu-0| I120: DISKLIB-LIB : Opened "/vmfs/volumes/557050be-########-####-##########3f/VMNAME/VMNAME.vmdk" (flags 0xa, type vmfs).

| vcpu-0| I120: DISK: Disk '/vmfs/volumes/557050be-########-####-##########3f/VMNAME/VMNAME.vmdk' has UUID '60 00 c2 97 fb 03 7d b1-58 b2 a5 88 31 a7 f8 7b'

vcpu-0| I120: DISK: OPEN '/vmfs/volumes/557050be-########-####-##########3f/VMNAME/VMNAME.vmdk' Geo (7832/255/63) BIOS Geo (0/0/0)

vcpu-0| I120: Creating virtual dev for scsi0:0

| vcpu-0| I120: DumpDiskInfo: scsi0:0 createType=11, capacity = 125829120, numLinks = 1, allocationType = 2

vcpu-0| I120: SCSIDiskESXPopulateVDevDesc: Using FS backend

| vcpu-0| I120: DISKUTIL: scsi0:0 : geometry=7832/255/63

vcpu-0| I120: VMXNET3 user: Ethernet0 Driver Info: version = 402822 gosBits = 2 gosType = 2, gosVer = 24848, gosMisc = 212

vcpu-0| I120: SnapshotVMXTakeSnapshotWork: Transition to mode 1.

vcpu-0| I120: SnapshotVMXTakeSnapshotComplete: Done with snapshot 'test': 0

vcpu-0| I120: SnapshotVMXTakeSnapshotComplete: Snapshot 0 failed: Error (3).

vcpu-0| I120: VigorTransport_ServerSendResponse opID=254F67FE-00000087-ce-3b-ab7a seq=6800308: Completed Snapshot request.

vcpu-0| I120: HBACommon: First write on scsi0:0.fileName='/vmfs/volumes/557050be-########-####-##########3f/VMNAME/VMNAME.vmdk'

vcpu-0| I120: DDB: "longContentID" = "ef8f023c4c6a95edaac40b89b58106af" (was "420f4698ff23a48a836fcf7908fb9968")

vcpu-0| I120: DISKLIB-CHAIN : DiskChainUpdateContentID: old=0x8fb9968, new=0xb58106af (ef8f023c4c6a95edaac40b89b58106af)

vmx| I120: SSL: EOF in violation of protocol

vmx| I120: SOCKET 2436 (132) recv error 0: Success

vmx| I120: Vix: [18917824 mainDispatch.c:2879]: VMAutomation: Connection Error (1) on connection 2433.

vmx| I120: VigorTransportProcessClientPayload: opID=254F67FE-00000112-9e-8f-b0d9 seq=6800667: Receiving Snapshot.Take request.

 

 

Environment

VMware vSphere ESXi

 

Cause

This issue occurs due to space issues on LUN and the underlying LUN will not have enough space.

Resolution

Ensure there is enough space on the datastore and underlying LUN on which the virtual machine resides:

 

1. Check the actual free space from command line by using the command:
   
   cd /vmfs/volumes/Datastore/
df -h
 
2. If an error occurs, check the LUN space by logging into the storage.
3. Perform storage vMotion and try to take snapshot.
4. If snapshot operation fails, try to increase the space on the datastore.

Additional Information

This is mainly noticed in thin provisioned LUNs

Do not perform storage vMotion if replication is going on for the VM else you will get the following message

 

Failed to copy one or more disks.

A fatal internal error occurred. See the virtual machine's log for more details.

Failed waiting for data. Error 195887107. Not found.