How to Transition HA heartbeating from VMFS datastores to vSAN and remove VMFS datastore

Products

• VMware vCenter Server

Issue/Introduction

When attempting to unmount or delete a decommissioned VMFS datastore from a vSphere cluster, the task may fail with a "Resource is in use" error even if the datastore is empty and hosts are in maintenance mode. This is often caused by a persistent vSphere High Availability (HA) heartbeat lock.

• The vCenter UI reports: "Unmount VMFS volume failed!"
• vpxd.log error: "Cannot unmount volume 'Datastore Name: VMFS uuid: ####-####-####-####' because file system is busy."
• vSphere HA primary node maintains a heartbeat lock on the datastore.

Environment

• VMware vSphere (all versions)
• VMware vSAN  (all versions)

Cause

 The vSphere HA (High Availability) service uses the datastore for heartbeating, which creates a lock within the hidden .vSphere-HA directory. This process keeps the file system busy, preventing the unmount operation.

Resolution

Phase 1: Update vSphere HA Heartbeat Configuration

Navigate to Cluster > Configure > vSphere Availability > Edit > Heartbeat Datastores.

Select "Automatically select datastores accessible from the host" or deselect <Datastore> from the  Available heartbeat datastores options. 

To prevent vSphere HA from throwing a permanent configuration warning ("The number of vSphere HA heartbeat datastores for this host is 1, which is less than 2") after removing the last VMFS datastore, add the following advanced option.

Under vSphere Availability > Advanced Options, add key das.ignoreinsufficienthbdatastore with value true, then click OK.

Reconfigure vSphere HA on the cluster to apply the new setting across all 13 ESXi hosts by turning HA off then back ON.

 

Phase 2: Pre-Unmount Prerequisites

Confirm no VMs, templates, or ISO images reside on <Datastore> for the <Cluster> in question.

Verify no host scratch partitions, syslog paths, or coredump files are hosted on <Datastore> for the <Cluster> in question.

 

Phase 3: Unmount and Detach Procedure

In the vSphere Client, right-click <Datastore>, select Unmount Datastore, and select all ESXi hosts for the <Cluster> in question.

Under Host > Configure > Storage Devices, select the underlying LUN for <Datastore> and execute a Detach across the ESXI hosts for the <Cluster> in question.

Perform a cluster-wide storage rescan to confirm the device status displays as unmounted/detached.

Notify the storage administration team that the LUN can now be safely unpresented/unmasked at the SAN array layer.  Once unmapped rescan storage and Datastore will no longer be present on cluster.