.vsan.stats namespace object not upgrading object version
search cancel

.vsan.stats namespace object not upgrading object version


Article ID: 326829


Updated On:


VMware vSAN


.vsan.stats namespace object may not upgrade from version 7 to 10 when upgrading to 6.7U3 or from version 13 to 14 when upgrading to version 7.0U2

A vCenter alert shows that a vSAN object is on an older version

From the command line of an ESXi node you see the following:

esxcli vsan debug object overview
Object UUID                           Group UUID                            Version  Size       Used     SPBM Profile                 Healthy Components
------------------------------------  ------------------------------------  -------  ---------  -------  ---------------------------  ------------------
c39b7e5d-aeca-7a0d-1ddf-005056016fbb  be9b7e5d-8fb8-ea92-ba64-005056016fbb       10  40.00 GB   0.01 GB  vSAN Default Storage Policy  3 of 3
be9b7e5d-8fb8-ea92-ba64-005056016fbb  be9b7e5d-8fb8-ea92-ba64-005056016fbb       10  255.00 GB  0.68 GB  vSAN Default Storage Policy  3 of 3
cbadc85b-9500-83ae-3be4-005056016fc3  cbadc85b-9500-83ae-3be4-005056016fc3       7  255.00 GB  1.36 GB  vSAN Default Storage Policy  3 of 3

 esxcli vsan debug object list|grep -E 'Object UUID:|Version:|Path:'|grep -ve objectVersion:
Object UUID: c39b7e5d-aeca-7a0d-1ddf-005056016fbb
   Version: 10
   Path: /vmfs/volumes/vsan:524cb6a621a5e774-f81d714149fc60df/be9b7e5d-8fb8-ea92-ba64-005056016fbb/.VM1.vmdk (Exists)
Object UUID: be9b7e5d-8fb8-ea92-ba64-005056016fbb
   Version: 10
   Path: /vmfs/volumes/vsan:524cb6a621a5e774-f81d714149fc60df/.VM1 (Exists)
Object UUID: cbadc85b-9500-83ae-3be4-005056016fc3
   Version: 7
   Path: /vmfs/volumes/vsan:524cb6a621a5e774-f81d714149fc60df/.vsan.stats (Exists)

Note: The preceding command excerpts are only examples. Environmental variables may vary depending on the customer's environment.


VMware vSAN 6.7.x
VMware vSAN 7.0.x


Option 1:
1) Change the storage policy by doing the following:
   a) Click on the cluster name in the left hand pane
   b) select Configure>vSAN Services
   c) Click on Edit to the left of Performance Service
   d) Select a different storage policy from the drop down list
   e) From RVC drilled down to the cluster level run vsan.ondisk_upgrade to start a manual ondisk upgrade
2) Run esxcli vsan debug object overview to confirm all objects are on version 10
3) If successful and the original Storage policy for Performance Service is desired, set it back

Note: Changing the storage policy can work for vmdks/namespaces as well that may have failed to upgrade to version 10

Option 2: 
If option 1 doesn't resolve the issue try the following:
Disable Performance Service and then re-enable, this will create a new .vsan.stats namespace with version 10

To disable vSAN Performance
1) Select the vSAN cluster
2) Select Configure Tab
3) Under vSAN select Services
4) Select Performance Service and click Edit
5) Click the toggle switch to disable performance service and click Apply

To enable vSAN Performance
1) Since you should already still be at vSAN Services
2) Click Edit to the right of Performance Service
3) Click the toggle switch to enable performance service and click Apply

Note: This will delete any perf stats already collected. Make sure to mention this to the customer in case they are in the middle of troubleshooting a performance issue and need to save the data.