Warning in vSAN Health Service - Stats DB Object - running out of free space

Products

• VMware vSAN

Issue/Introduction

The vSAN Stats DB object is running low on free logical disk space. This check does not take into account actual physical free space on the datastore. The vSAN Stats DB object is limited to 255 GB.  If the utilization reaches above 80 percent, it enters a yellow waring state and if over 95 it enters a Red error state. History data in the Stats DB object is kept for approximately three months (93 days), and old data is purged. 

Example

 

 

 

 

 

Environment

VMware vSAN (all versions)

Cause

The collection interval within the vsanperf.conf  file has  been changed from the default value of 300 leading to increased file size. 

Example

 In the example below the 

[VSANPERF]
port = 9095
metrics_port = 9097
namespace = /vsanperf
keyfile = /etc/vmware/ssl/rui.key
certfile = /etc/vmware/ssl/rui.crt
ca_certs = /etc/vmware/ssl/castore.pem
cert_reqs = True
no_auth = False
interval = 30    <------------ value has been changed from 300 
heart_beat = 5
max_workers = 4
loglevel = info
syslogident = VSANMGMTSVC
logsize = 10
logrotate = 8
data_keep_days = 93
db_purge_debug_mode = False
config_keep_limit = 5000
compress_config_info = 50
use_ramdisk = 0
diagnostic_mode = False
diagnostic_mode_ramdisk = 50
diagnostic_mode_interval = 1
diag_mode_duration = 1800
diag_mode_expiration = 86400
vsantop_mode = False
vsantop_mode_ramdisk = 100
vsantop_mode_interval = 5
ioinsight_mode = False
ioinsight_mode_ramdisk = 100
ioinsight_mode_interval = 10
ioinsight_data_keep_days = 90
top_n_size = 50

 

 

Resolution

Follow the action below to correct this issue. 

1. Review the etc/vmware/vsan/vsanperf.conf File on each host in the vSAN cluster and determine if the interval has been changed from 300.
   
   cat etc/vmware/vsan/vsanperf.conf | grep -i interval
   
   
2. Set the perf stat interval back to default settings of 300 via the following command on all hosts that have been changed.
   
   esxcli vsan perf set --interval=300
   
   
3. Disable then reenable the vSAN Performance service to recreate the vsan.stats DB object.

If you are on vCenter 8 follow KB Unable to Disable Performance Service in vSAN 8 to disable and enable the performance service

Reference KB

Error in vSAN Health Service - Not enough free space for vSAN performance database object