Enabling the VMware vSphere Storage I/O Control log might result in flooding syslog and rsyslog servers

Products

• VMware vSphere ESXi

Issue/Introduction

This article provides information about causes and solution of VMware vSphere Storage I/O Control log flooding syslog and rsyslog servers.

Symptoms:

• Enabling VMware vSphere Storage I/O control log results in flooding syslog and rsyslog servers like VMware Log Insight in vSphere 6.7
• Esxi hosts logging excessively in /var/log/storagerm.log

In the /var/log/storagerm.log, you see entries similar to:
 

2019-07-17T13:39:55Z storageRM[3960703]: World id 2350022: IOcount= 3 avgOIO= 0.00 demandOIO= 0 reservation= 4 throughputCap= 2147483647 shares= 8000 iops= 3 avgLat= 0 reservationVio= 0 avgIOSz= 7716
2019-07-17T13:39:55Z storageRM[3960703]: World id 3702929: IOcount= 5 avgOIO= 0.00 demandOIO= 0 reservation= 4 throughputCap= 2147483647 shares= 8000 iops= 5 avgLat= 0 reservationVio= 0 avgIOSz= 14113
2019-07-17T13:39:55Z storageRM[3960703]: datastoreName: Host shares: 16000 OIO based shares: 2 currentBeta = 0.00 originalBeta = 4.83
2019-07-17T13:39:55Z storageRM[3960703]: datastoreName, 0
2019-07-17T13:39:55Z storageRM[3960703]: datastoreName, 0
2019-07-17T13:39:55Z storageRM[3960703]: datastoreName, 0
2019-07-17T13:39:55Z storageRM[3960703]: Sleep delta = 4239.910889, Sleep Time = 4000

 

In the /var/log/vmkernel.log, you see entries similar to:
 

2019-07-17T12:11:03.010Z cpu37:2501302)MemSchedAdmit: 489: uw.2501302 (2360636) extraMin/extraFromParent: 256/256, sioc (796) childEmin/eMinLimit: 14311/14336
2019-07-17T12:11:03.010Z cpu37:2501302)MemSchedAdmit: 471: Admission failure in path: sioc/storageRM.2501302/uw.2501302

 

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment

Environment

VMware vSphere ESXi 6.7

Cause

The root cause is a memory leak in storageRM

Resolution

 

This issue is resolved in ESXi 6.7 Patch 01, available at Download Patches . For more information, see the VMware ESXi 6.7, Patch Release ESXi670-201912001