VCSA CPU usage has stayed at 100% for several hours.

Products

• VMware vCenter Server

Issue/Introduction

• A large number of 'postgres: vc VCDB [local] FETCH' processes occurred during this issue occurring.
  $ grep -ic "vc VCDB \[local\] FETCH" procstate* procstate
  procstate-yyyymmdd:***
  procstate-yyyymmdd:*** <=== A large number 
  
  The procstate-yyyymmdd file shows a large number of 'postgres: vc VCDB [local] FETCH' entries.
  ========== Start of system state dump at *** ==========
  /bin/ps auxww
  USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
  vpostgr+   ***   ***        ***           ***      ***        ?       Rs           ***        *** postgres: vc VCDB [local] FETCH
• SQL queries related to Task and Event with a long duration time were executed.
  postgresql-***.log
yyyy-mm-ddThh:mm:ss,nnn UTC *** 0 VCDB vc LOG:  duration: *** ms  statement: BEGIN;declare "SQL_CUR***" cursor withhold for  SELECT TASK_ID, NAME, DESCRIPTIONID, ENTITY_ID, ENTITY_TYPE,
yyyy-mm-ddThh:mm:ss,nnn UTC *** 0 VCDB vc LOG:  duration: *** ms  statement: BEGIN;declare "SQL_CUR***" cursor withhold for  SELECT EVENT_ID, CHAIN_ID, EVENT_TYPE, EXTENDED_CLASS, CREATE_TIME, 
• The value of mem_used_bytes increased during the time of this issue occurring compared to other periods.
  postgresql-***.log
yyyy-mm-ddThh:mm:ss,nnn UTC *** 0   LOG:  Memory check: flag = rss, mem_used_bytes = *** kB, mem_avail = *** kB
• An internal solution user, such as vpxd-extension-***, executed SQL.
  vpxd-profiler-***.log
--> /SessionStats/SessionPool/Id='***'/Username='VSPHERE.LOCAL\vpxd-extension-***'/PropertyCollector/ComputeGUReqTime/max 0
• The database was configured to retain Task and Event data for a longer duration, instead of the default 30 days.
  vSphere Client > the target vCenter Server > Configure > Settings > General > Database > Task retention (days) and Event retention (days)

Environment

VMware vCenter Server 7.*
VMware vCenter Server 8.*

Cause

The Task and Event data stored in VCDB for a long period of time may have caused SQL execution to take a long time.

Resolution

To consider deleting old task and event data by referring to the article, "Delete old tasks, events and statistics data in vCenter Server 5.x, 6.x, 7.x and 8.x.".

Additional Information

vmware-vpostgres server 100% CPU Usage in vCenter server
VCSA Performance issue or high CPU usage after upgrading from 7.0u2 to 7.0u3