Deleting vSAN Stats DB Object in RVC Fails - VMware vSAN

Products

• VMware vSAN

Issue/Introduction

The vSAN Performance Service may encounter a "Stats primary election" error, or management commands in RVC may fail when the .vsan.stats object becomes corrupted or enters an orphaned state. This guide provides steps to diagnose this issue and perform a forced cleanup of the object.

Environment

• VMware vSAN 8.x

• Affected environments: 4-node vSAN clusters, typically occurring after wiping drives or reinstalling ESXi nodes.

Cause

This issue occurs when the .vsan.stats object is corrupted, orphaned, or when the advanced configuration parameter /VSAN/DOMPauseAllCCPs is incorrectly set to 1.

Resolution

1. Check and Reset Advanced Parameters

1. 1. Log in to each ESXi host in the affected vSAN cluster via SSH.
   2. Run the following command to check the current value: esxcfg-advcfg -g /VSAN/DOMPauseAllCCPs
   3. If the value is 1, reset it to 0: esxcfg-advcfg -s 0 /VSAN/DOMPauseAllCCPs
   4. Verify /VSAN/IgnoreClusterMemberListUpdates is also set to 0: esxcfg-advcfg -g /VSAN/IgnoreClusterMemberListUpdates
   5. If the value returns 1, reset it: esxcfg-advcfg -s 0 /VSAN/IgnoreClusterMemberListUpdates

2. Attempt RVC Cleanup

   1. Connect to the RVC console: rvc [email protected]@localhost
   2. Navigate to the vSAN cluster path: cd /localhost/<datacenter>/computers/<cluster>/
   3. Attempt to delete the stats object: vsan.perf.stats_object_delete .

3. Perform Forced Cleanup (If RVC fails) If RVC commands fail, perform a manual cleanup via the ESXi CLI:

   1. Identify the stale object UUID: esxcli vsan debug object list --all | grep -B1 "Directory Name: .vsan.stats"
   2. Force-delete the object using the identified UUID (e.g., ####): /usr/lib/vmware/osfs/bin/objtool delete -u #### -f
   3. Restart management services on all hosts: /etc/init.d/vsanmgmtd restart
   4. Re-enable the Performance Service in vCenter (Cluster > Configure > vSAN > Services).