vSAN datastore consumption increasing rapidly due to "Other improved virtual disks"

Products

• VMware vSphere Kubernetes Service

Issue/Introduction

• The vSAN capacity breakdown shows that the "Other improved virtual disks" category is consuming most space and increasing rapidly. 
  
  
• There is a mismatch in the number of PVs on the supervisor and the entries in the cns.volume_info table on the VCDB. 
  • Get the count of PVs on the supervisor:
    k get pv -A | wc -l
    
    
  • Get the count of entries in the VCDB:
    select COUNT(*) from cns.volume_info;

Environment

• vSphere Kubernetes Service
• Cloud Native Storage

Cause

Multiple stale/orphaned First Class Disks (FCDs) are present on the backend datastore and registered in the vCenter Database (VCDB), but they lack any corresponding Persistent Volumes (PVs) or Persistent Volume Claims (PVCs) on the Supervisor cluster.

Resolution

Contact Broadcom Support.