[Draft] Upgrading host from 8.0 U2 using stateful auto deploy boot fails to complete with error "cannot be unmounted. Reason: Busy"

Products

• VMware vCenter Server

Issue/Introduction

• Issue is seen post upgrading the host to ESXi 8.0 Update 2
• in /var/run/log/syslog.log the following entries can be found:

  <timestamp> Wa(12) hostprofile[2099609]: opID=MainThread: Core dump file is enabled. Disabling it
  <timestamp> Wa(12) hostprofile[2099609]: opID=MainThread: Shutting off coredump file: 0:True
  <timestamp> In(14) hostprofile[2099609]: opID=MainThread: Now caching to disk: t10.ATA_____#########################________________________###########
  <timestamp> Er(11) hostprofile[2099609]: opID=MainThread: EngineModule::ApplyHostConfig. Exception: ("Errors: \nVolume '########-########-####-############' cannot be unmounted. Reason: Busy\n", b'')
  <timestamp> Wa(12) hostprofile[2099609]: opID=MainThread: EngineModule::ApplyHostConfig. Backtrace:
  <timestamp> Wa(12)[+] hostprofile[2099609]:
  <timestamp> Wa(12) hostprofile[2099609]: opID=MainThread:    File "/lib64/python3.8/site-packages/hostprofiles/tests/tools/hpcliModules/engineModule.py", line 553, in ApplyTaskList
  <timestamp> Wa(12)[+] hostprofile[2099609]:
  <timestamp> Wa(12) hostprofile[2099609]: opID=MainThread:    File "/lib64/python3.8/site-packages/hostprofiles/pyEngine/applyConfigSpec.py", line 4724, in ApplyHostConfig
  <timestamp> Wa(12) hostprofile[2099609]: opID=MainThread: Adding missing key EsxcliError.summary
  <timestamp> Wa(12) hostprofile[2099609]: opID=MainThread: excObj is of unknown type: <class 'esxutils.EsxcliError'>
  <timestamp> In(14) hostprofile[2099609]: opID=MainThread: Cleaned up Host Configuration
  <timestamp> In(14) hostprofile[2099609]: opID=MainThread: /sbin/applyHostProfile sending boot msg to console: Applying Host Profile task list...Error
  <timestamp> Er(11) hostprofile[2099609]: opID=MainThread: while applying tasks: (vmodl.fault.SystemError) {

 

Environment

VMware vSphere ESXi 8.0.2

Cause

When the chosen system cache partition in use, it cannot be reformatted/unmounted. The stateful install or system cache fails due to one of the following causes:

1. the partition is used by OSData 
2. A log file is hardcoded to some file path of that partition

Resolution

Broadcom is working towards a permanent fix for this issue.

In order to workaround the issue, implement one of the below workaround

• Log in to the vCenter server using UI
• Navigate to the desired cluster
• Click Configure on the right hand pane -> select vSphere Availability -> Edit
• Go in Edit Cluster Settings and click Advanced Options
• Add the following Option:

Option: das.config.fdm.dumpDirectory
Value:   /var/log

• Click Ok
• Perform a reconfigure HA on all the hosts in the cluster

 

Additional Information

Note: Do not disable vSphere HA if there are FT (Fault Tolerance) VM running on the cluster