High disk latency on VMs and hard resets with "Failed to issue sync i/o : Busy" PVSCSI error
Article ID: 435544
Updated On:
Products
Issue/Introduction
The virtual machine experiences intermittent "hard resets" and performance degradation. Analysis of vmware.log and hostd logs reveals the following indicators:
Error Message: vmkernel.log: PVSCSI: 2682: Failed to issue sync i/o : Busy
Error Message: Hostd.log:No VMkernel vSCSI handler for disk scsi0:X (ctrl:1000, path:...........
Guest Behavior: VM vmware.log: The Guest OS SCSI timeout is reached, leading to a "hard reset" request to the virtual hardware.
Environment
ESXi 8.x
Cause
The virtual machine's performance is bottlenecked because all high-intensity data disks are routed through a single PVSCSI Controller (SCSI 0). This configuration forces the database and Operating System's concurrent I/O requests to share a limited virtual execution path and queue depth, quickly exhausting the controller's capacity. The resulting synchronous I/O failures are misinterpreted by the Guest OS as a hardware hang, which initiates a system reset for recovery.
Resolution
Optimizing I/O Performance by Distributing Load with Multiple PVSCSI Controllers
The recommended strategy to eliminate I/O bottlenecks and significantly improve throughput is to distribute the I/O load across several virtual SCSI controllers. This method achieves better performance by creating independent queues and enabling parallel interrupt processing.
1. Add Virtual Controllers
- Power off the Virtual Machine (VM).
- Access the VM Settings and edit them.
- Add the necessary number of additional VMware Paravirtual SCSI (PVSCSI) controllers.
2. Redistribute Virtual Disks
- Re-assign the existing virtual disks to the new SCSI controllers to ensure the I/O load is evenly balanced.
3. Operating System Verification
- Linux: Modern kernels will automatically detect the new device paths. To prevent boot issues or mount failures due to changing device names (e.g., /dev/sdb to /dev/sdd), confirm that the /etc/fstab configuration utilizes persistent identifiers like UUID or LABEL.
- Windows: The operating system will automatically detect the new controllers and install the required PVSCSI drivers. A system reboot may be necessary to complete the drive mapping and finalize the configuration.
Additional Information
Feedback
