Virtual Machines on vSAN 7.x Using Independent Non-Persistent Delta and Related Redo vmdks May Become Inaccessible After Shutdown

Products

• VMware vSAN

Issue/Introduction

This KB is written to inform of this issue, and to provide workaround steps and solution.

Symptoms:
vSAN 7.x is in use
Virtual Machine is using an independent non-persistent delta and related redo vmdks (typically in a VDI environment such as Citrix or Horizon)
When a VM is powered off it enters an inaccessible state from the UI (will show as invalid from host command line with vim-cmd vmsvc/getallvms)


When viewing the virtual Objects in the UI for VMs you will see them as Inaccessible.


Example configuration - your dates, times, UUIDs, names, and other details will vary:
Disks listed in vmx :

scsi0:1.deviceType = "scsi-hardDisk"
scsi0:1.fileName = "disk_IdentityDisk.vmdk"
sched.scsi0:1.shares = "normal"
sched.scsi0:1.throughputCap = "off"
scsi0:1.present = "TRUE"
scsi0:2.deviceType = "scsi-hardDisk"
scsi0:2.fileName = "disk_TemporaryStorage.vmdk"
sched.scsi0:2.shares = "normal"
sched.scsi0:2.throughputCap = "off"
scsi0:2.present = "TRUE"
scsi0:0.deviceType = "scsi-hardDisk"
scsi0:0.fileName = "disk-8aaf21aa-f59a-4095-82b8-a4b51200d7f3-xd-delta.vmdk"
scsi0:0.mode = "independent-nonpersistent"

VM's vmware.log file towards power off, this is a normal and expected series of messages:

2021-11-30T23:17:49.415Z In(05) vmx - DISK: Discarding redo log for nonpersistent disk scsi0:0: ./disk-8aaf21aa-4792-4095-6548-a4b51200d7f3-xd-delta.vmdk.REDO_zfs0nu
2021-11-30T23:17:49.418Z In(05) vmx - DISKLIB-VMFS : "vsan://528d96d33daf3851-5ed111625fae60fc/a6f1a561-8087-3c76-3453-e4434bda8196" : open successful (1114113) size = 0, hd = 0. Type 20
2021-11-30T23:17:49.418Z In(05) vmx - DISKLIB-VMFS : "vsan://528d96d33daf3851-5ed111625fae60fc/a6f1a561-8087-3c76-3453-e4434bda8196" : closed.
2021-11-30T23:17:49.419Z In(05) vmx - OBJLIB-VSANOBJ: VsanObjUnlink: Unlinking '/vmfs/volumes/vsan:528d96d33daf3851-5ed111625fae60fc/567c1f5f-9c44-5ec5-1bd2-e4434b736d96' -- 'vsan://528d96d33daf3851-5ed111625fae60fc/a6f1a561-8068-3b77-3563-e4434bda8196'.
2021-11-30T23:17:49.447Z In(05) vmx - DISKLIB-LIB : DiskLibUnlinkInt: Disk delete successfully completed { result:0, Msg: 'The operation completed successfully', fileName:'./disk-8aaf21aa-4792-4095-6548-a4b51200d7f3-xd-delta.vmdk.REDO_zfs0nu'}

In hostd.log you see the following:

2021-11-30T23:18:10.925Z error hostd[2103066] [Originator@6876 sub=Vmsvc.vm:/vmfs/volumes/vsan:528d96d33daf3851-5ed111625fae60fc/e77de15f-f122-3b21-e868-e4434d739b50/vmname.vmx opID=HB-SpecSync-host-77142@1091845-1c915dac-74-348b user=vpxuser] Invalid transition requested (VM_STATE_INVALID_LOAD -> VM_STATE_RECONFIGURING): Invalid state

Environment

VMware vSAN 7.0.x

Cause

When a VM with an independent non-persistent disk is powered on a shadow REDO disk is created for the non-persistent disk to track changes.
When the VM is powered off the non-persistent disk is deleted along with the related REDO disk.
The REDO disk vSAN object is registered with the vSAN APD (All Paths Down) monitor.
vSAN APD monitor reports the disk is not accessible and the VM transitions to Inaccessible/Invalid.
The reference to the REDO disk is removed from the VM and the VM is able to be transitioned back to accessible state, but this takes manual intervention as listed in the workaround section.

Resolution

This issue will be resolved in an upcoming vSAN 7.x version

Workaround:
Unregister the virtual machine on the host command line or the UI
Register the virtual machine on a host

Additional Information

Impact/Risks:
Virtual machines need to be unregistered and reregistered to power back on