Configuring ESXi 8 Stateless Image Caching Fails with "Volume '<UUID>' cannot be unmounted. Reason: Busy" Due to Audit Logging

Products

• VMware vCenter Server

Issue/Introduction

• When configuring stateless image caching on an ESXi 8 host, the operation fails with the following error:
  Volume '<volume_UUID>' cannot be unmounted. Reason: Busy


• Log file /var/run/log/vmkernel.log will show similar entries to:
  WARNING: VC: 2445: unmounting opened volume ('<UUID>' '<DatastoreName>') with ref 3 is not allowed.
FSS: 9877: File audit.014, fhID: 6685457, is opened by world: 000002098025 - vmsyslogd
VC: 2716: Unmount volume ... : Busy


• Output from /tmp/openfile-before-unmount.txt shows audit.014 belongs to world - syslog:
  grep audit.014 openfile-before-unmount.txt
2098025     vmsyslogd             FILE                        9   /vmfs/volumes/'<UUID>' '<DatastoreName>'/auditLog/audit.014

Environment

vSphere ESXi 8.x

Cause

During stateless caching configuration, ESXi attempts to repartition the system disk, which includes temporarily relocating the scratch partition to /tmp/scratch (a ramdisk).

If audit logging to local storage is enabled, the vmsyslogd process keeps audit log files open on the current scratch volume (/scratch/auditlog by default). Since audit logs are not allowed on volatile memory like /tmp, they remain active on the persistent disk, preventing it from being unmounted.

As a result, the caching operation fails with a "Volume busy" error.

Resolution

To resolve the issue, temporarily disable audit recording to local storage before configuring stateless image caching. Once caching is successfully applied, audit logging can be re-enabled.

Option 1: Using esxcli commands

• Disable audit logging to local storage:
  esxcli system auditrecords local disable
  
  
• Apply stateless image caching via Host Profile remediation.
  
  
• Re-enable audit logging:
  esxcli system auditrecords local enable

Option 2: Using Host Profiles

• Set Syslog.global.auditRecord.storageEnable to false in the host profile and remediate the host.
  This disables audit logging to local storage.

• Configure System Image Cache settings in the host profile and remediate the host.
  The caching should now proceed without error.

• Set Syslog.global.auditRecord.storageEnable to true in the host profile and remediate again to re-enable audit logging.

Additional Information

Ensure that any automation frameworks using host profiles include this toggle if audit logging is enabled by default.