Excessive logging by iavmd driver may result in slow operations

Products

• VMware vSAN

Issue/Introduction

Symptoms:

• Operations may run slowly when the iavmd driver is in use on the ESXi host
• The var/log/vmkernel.log files may show errors, similar to:

2020-03-09T23:47:53.445Z cpu25:2551715)nvme_ReadError
2020-03-09T23:47:53.446Z cpu2:2551717)nvme_ReadError
2020-03-09T23:47:53.446Z cpu33:2551718)nvme_ReadError

Environment

VMware vSAN 7.0.x

Cause

This issue occurs due to the iavmd driver's default logging level is DEBUG, slowing the backup process.

This is a driver issue in iavmd-2.0.0.1055-4 and earlier versions.

Resolution

This issue is resolved in version iavmd-2.0.0.1055-5 and later of the iavmd driver.

Workaround:
Run these commands on the affected ESXi hosts to adjust the logging level:

• vsish -e set /system/modules/iavmd/loglevels/intel-nvme 4
• vsish -e set /system/modules/iavmd/loglevels/intel-nvme-vmd 4