A general system error occurred: Too many outstanding operations

Products

• VMware vSphere ESXi
• VMware vCenter Server

Issue/Introduction

VM Snapshot tasks fail in vCenter Server with the below errors,

A general system error occurred: Too many outstanding operations

Most tasks that are run on a specific host fail with

A general system error occurred: Too many outstanding operations

or

Operation Timed Out

Esxcli commands that are run on the console also fail with the error

failed: A general system error occurred: Too many outstanding operations

Environment

VMware vCenter Server

VMware vSphere ESXi

Cause

The management services on the ESXi host have a lot of queued tasks.

hostd.log has the below error:

info hostd[00000] [Originator@6876 sub=Solo.Vmomi opID=2abe172e-f7-c5e8 user=vpxuser Result:  
--> (vmodl.fault.SystemError) {  
-->    reason = "Too many outstanding operations",  
-->    msg = "",  
--> }  

Review current tasks on the esxi host using the command, vim-cmd vimsvc/task_list

Are there are a lot of pending tasks?

Resolution

To terminate the pending tasks and to resolve the issue, restart the management services on the ESXi host. See Restarting the Management agents on an ESXi or ESX host.

Review the article, Collecting information about tasks in VMware ESXi for more information on pending tasks on the esxi host.

Review performance of the esxi host. Are there any performance issues that could cause hostd to excute tasks slowly?
Specifically review I/O performance of the boot disks and the datastores backing the VMs.