Identify and resolve issues with ESXi host(s) hitting a vsanmgmt memory leak.
Symptoms:
You experience these symptoms on an ESXi 7.0 U1 or 7.0U2 vSAN enabled host
- Physical disk health retrieval issues with error Host XXX - Error: Out of Memory
- Advanced vSAN Configuration in sync - vSAN daemon liveness - CLOMD status: Abnormal
- Inaccurate health check results due to vsanmgmtd memory leak
In the
/var/run/log/vmkernel.log files on the ESXi host, you see messages similar to:
- 2021-01-20T09:33:12.993Z cpu40:13487007)Admission failure in path: host/vim/vmvisor/vsanperfsvc:cmmds-tool.13487007:uw.13487007
- 2021-01-20T09:33:12.993Z cpu40:13487007)Admission failure in path: host/vim/vmvisor/vsanperfsvc:cmmds-tool.13487007:uw.13487007
In the
var/run/log/vsanmgmt.log file, you see messages similar to:
- 2021-01-20T03:33:55.754Z info vsand[2510678] [opID=MainThread statsdaemon::_logDaemonMemoryStats] Daemon memory stats: eMin=171.324MB, eMinPeak=173.968MB, rMinPeak=174.984MB MEMORY PRESSURE
- 2021-01-20T04:03:59.304Z info vsand[2510678] [opID=MainThread statsdaemon::_logDaemonMemoryStats] Daemon memory stats: eMin=170.428MB, eMinPeak=173.968MB, rMinPeak=174.984MB MEMORY PRESSURE
In the
/var/run/log/vmsyslogd-dropped.log file, you see messages similar to:
- 2020-09-25T04:38:38.257Z: 2020-09-25T04:38:38.021Z cpu51:18265550 opID=34b30f7d)Admission failure in path: host/vim/vmvisor/vsanperfsvc:vsanmgmtd.18265531:uw.18265531
Note: The timestamp and memory usage will vary for different clusters.
The vsanmgmtd core dump file (
vsanmgmtd-zdump.XXX) can be found under
/var/core/