Task error "Retrieve a virtual storage object" and "Sync volume" are not stopping.

Products

• VMware vCenter Server

Issue/Introduction

•  This is a k8s deployment.
•  The datastore shared across vCenter Server instances.
•  In the journalctl log, the following log repeatedly occurs.
    [vim.event.TaskEvent] [info] [com.vmware.cns] [] [***] [Task: Retrieve a virtual storage object]
    [vim.event.TaskEvent] [info] [com.vmware.cns] [] [***] [Task: Sync volume]
  
  
•  In the vpxd log, failed lock vmdk file log repeatedly occurs.
    yyyy-mm-ddThh:mm:ss.nnn error vpxd ERROR VStorageObjectManager -- vim.vslm.vcenter.VStorageObjectManager.retrieveVStorageObjectMetadata: :vmodl.fault.SystemError
    --> Result:
    --> (vmodl.fault.SystemError) {
    -->    faultCause = (vmodl.MethodFault) null,
    -->    faultMessage = <unset>,
    -->    reason = "Failed to lock the file: api = DiskLib_Open, _diskPath->CValue() = /vmfs/volumes/***/fcd/***.vmdk"

Environment

VMware vCenter Server 

Cause

No datastore should be visible to more than 1 vcenter. This is the product assumption of CNS/CSI stack.

Resolution

To make the datastore available on a single vCenter Server.
For details on how to remove a datastore from ESXi, refer to "How to detach a LUN device from ESXi hosts."

How to detach a LUN device from ESXi hosts
https://knowledge.broadcom.com/external/article/323128/

Additional Information

"vSphere Container Storage Plug-in does not support datastores shared across vCenter Server instances."

vSphere Functionality Supported by vSphere Container Storage Plug-in
https://techdocs.broadcom.com/us/en/vmware-cis/vsphere/container-storage-plugin/3-0/getting-started-with-vmware-vsphere-container-storage-plug-in-3-0/vsphere-container-storage-plug-in-concepts/vsphere-functionality-supported-by-vsphere-container-storage-plug-in.html