Operations sent to CNS returned a task id, but the task state never changes from queued. The user knows these tasks are for volumes attached to a virtual machine that is just deleted.
CNS recently introduced new serialization logic to schedule tasks conflict-free. This relies on virtual machine attachment information to be accurate. When the virtual machine is deleted before all its volumes operations can be scheduled, such attachment information is lost and confuses CNS scheduling.
The issue is not fixable from CNS perspective. To avoid this issue, please wait for all the volumes operations to be completed before deleting the Virtual Machine.
To workaround the issue, please follow the below mentioned steps:
If application layer can fix the calling order, nothing needs to be done on CNS side. If not, CNS new serialization can be disabled in below steps:
Open /usr/lib/vmware-vsan/VsanVcMgmtConfig.xml on vCenter.
Change below config to false.
<newSerializationEnabled>true</newSerializationEnabled>
Restart vsan-health with 'vmon-cli -r vsan-health'.