Alarm not return to green automatically when more than 50 virtual machines running in one ESXi host

Products

• VMware vCenter Server 7.0
• VMware vCenter Server 8.0
• VMware vSphere ESXi 7.0
• VMware vSphere ESXi 8.0

Issue/Introduction

• There are more than 50 virtual machines running in the ESXi host.
• CPU or memory usage alarm happens in these virtual machines or the ESXi host.
• Before the alarm condition cleared there are changes such as new virtual machine migrated to or from the ESXi host.
• The alarm will not return to green automatically even through trigger condition disappears. 

Environment

VMware vSphere 7.0.x
VMware vSphere 8.0.x

Cause

VPXA is set to process no more than 100 (50*2) changes at a time. Whenever there are more changes than that, the first ones in the batch are being erased and therefore, vpxd does not receive the notification for those changes. Alarm triggered before these changes will not be able cleared automatically.

Resolution

This is a known issue and resolved in vSphere 8.0 U3. 
Upgrade both vCenter Server and ESXi to 8.0 U3 or later.