Cannot apply host configuration as the host profile remediation fails with error: "Host profile apply failed with error: Cannot remove datastore"

Products

• VMware vCenter Server
• VMware vSphere ESXi

Issue/Introduction

• Error observed in the vSphere client - 'Apply host configuration' task:

Host profile apply failed with error: Cannot remove datastore '<datastore-name>' because "One or more virtual machines are still registered on it". Correct the problem and retry the operation.

• /var/run/log/vmkernel.log snippets reports similar to:

WARNING: NFS: 3113: <datastore-name> has open files, cannot be unmounted.

Cause

When a storage-related settings in the Host Profile (like a multipathing policy, NFS version, mount option, mount point name, or even a custom attribute) needs to be changed, vSphere treats that as a significant config update. To apply the changes safely, the host unmounts that datastore, applies the updated configuration and remounts the datastore.

If the datastore is being used by the virtual machines running on the host or the host itself (as heartbeat datastore), the unmount operation fails as the datastore is being used.  

Resolution

• Validate the Host Profile to the respective storage configuration on the host.
• In the below example, the NFS datastore configuration 'Vmknic name of the mount point' is null (--). While the NFS datastore mounted on the host had the value of 'Vmknic name of the mount point" as "vmknic1".

• To effectively compare differences between host configurations, extract the host profile from the ESXi host targeted for remediation. Then, compare the storage configuration settings with those in the problematic host profile.
• In this example, focus on the NFS datastore configuration, specifically, the 'Vmknic name of the mount point', which should have an assigned value.
• To ensure consistency, edit the host profile attached to the affected host and manually add or remove the corresponding value to the storage configuration. Save the changes. This ensures that the host does not attempt to re-mount the datastore during the remediation process.