Re-signature operation of snapshot LUN is failing with fault: VmfsAlreadyMounted.

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

Symptoms:
Re-signature operation is failing with VmfsAlreadyMounted exception even though its original copy is not mounted/force mounted anywhere in the datacenter.
 

• Mounting a snapshot LUN from vCenter UI fails with an error:

"VMFS volume with the same uuid is already mounted in the datacenter"

• In vCenter vpxd. log reports following errors:

[VpxLRO] -- ERROR task-668821 -- datastoreSystem-80 -- vim.host.DatastoreSystem.resignatureUnresolvedVmfsVolume: vim.fault.VmfsAlreadyMounted:
--> (vim.host.UnresolvedVmfsResignatureSpec) {

-- ERROR task-668821 -- datastoreSystem-80 -- 
vim.host.DatastoreSystem.resignatureUnresolvedVmfsVolume: vim.fault.VmfsAlreadyMounted:
--> Result:
--> (vim.fault.VmfsAlreadyMounted) {
-->    faultCause = (vmodl.MethodFault) null,
-->    faultMessage = <unset>,
-->    uuid = ""
-->    msg = ""
--> }
--> Args:
-->
--> Arg resolutionSpec:
--> (vim.host.) {
-->    extentDevicePath = (string) [
-->       "/vmfs/devices/disks/naa.xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx:1"
-->    ]

• Re-signature via esxcli works fine from ESXi host.
• Volume is not mounted on DR site or anywhere else as mentioned in KB: https://kb.vmware.com/s/article/2082643

Environment

VMware vCenter Server 6.5.x
VMware vCenter Server Appliance 6.7.x
VMware vCenter Server Appliance 6.5.x
VMware vCenter Server 6.7.x
VMware vSphere ESXi 6.5
VMware vSphere ESXi 6.7

Cause

The UnresolvedVmfsExtent returned by API QueryUnresolvedVmfsVolumes (from host side) is not filled with the vmfsUuid field. Issue occurs only if datacenter has at least one force mounted volume.

Resolution

This issue is resolved in vCenter Server 6.5 Update 3f (build number 15259038) patch release. To download go to VMware Patch Download Center  
This issue is resolved in vCenter Server 6.7 Update 3l (build number 17138064) patch release. To download go to VMware Patch Download Center  

Workaround:

• Re-signature the snapshot volume from ESXi host directly using esxcli:

esxcli storage vmfs snapshot resignature -l label|-u uuid

• For more details on mounting snapshot volumes, please see following KB and section:Command line:

           vSphere handling of LUNs detected as snapshot LUNs

Additional Information

Impact/Risks:
None.