.vsan.stats namespace object not upgrading object version
Article ID: 326829
Updated On:
Products
VMware vSAN
Issue/Introduction
Symptoms:
.vsan.stats namespace object may not upgrade from version 7 to 10 when upgrading to 6.7U3 or from version 13 to 14 when upgrading to version 7.0U2
A vCenter alert shows that a vSAN object is on an older version
From the command line of an ESXi node you see the following:
esxcli vsan debug object overview
Object UUID Group UUID Version Size Used SPBM Profile Healthy Components
------------------------------------ ------------------------------------ ------- --------- ------- --------------------------- ------------------
c39b7e5d-aeca-7a0d-1ddf-005056016fbb be9b7e5d-8fb8-ea92-ba64-005056016fbb 10 40.00 GB 0.01 GB vSAN Default Storage Policy 3 of 3
be9b7e5d-8fb8-ea92-ba64-005056016fbb be9b7e5d-8fb8-ea92-ba64-005056016fbb 10 255.00 GB 0.68 GB vSAN Default Storage Policy 3 of 3
cbadc85b-9500-83ae-3be4-005056016fc3 cbadc85b-9500-83ae-3be4-005056016fc3 7 255.00 GB 1.36 GB vSAN Default Storage Policy 3 of 3
esxcli vsan debug object list|grep -E 'Object UUID:|Version:|Path:'|grep -ve objectVersion:
Object UUID: c39b7e5d-aeca-7a0d-1ddf-005056016fbb
Version: 10
Path: /vmfs/volumes/vsan:524cb6a621a5e774-f81d714149fc60df/be9b7e5d-8fb8-ea92-ba64-005056016fbb/.VM1.vmdk (Exists)
Object UUID: be9b7e5d-8fb8-ea92-ba64-005056016fbb
Version: 10
Path: /vmfs/volumes/vsan:524cb6a621a5e774-f81d714149fc60df/.VM1 (Exists)
Object UUID: cbadc85b-9500-83ae-3be4-005056016fc3
Version: 7
Path: /vmfs/volumes/vsan:524cb6a621a5e774-f81d714149fc60df/.vsan.stats (Exists)
Note: The preceding command excerpts are only examples. Environmental variables may vary depending on the customer's environment.
.vsan.stats namespace object may not upgrade from version 7 to 10 when upgrading to 6.7U3 or from version 13 to 14 when upgrading to version 7.0U2
A vCenter alert shows that a vSAN object is on an older version
From the command line of an ESXi node you see the following:
esxcli vsan debug object overview
Object UUID Group UUID Version Size Used SPBM Profile Healthy Components
------------------------------------ ------------------------------------ ------- --------- ------- --------------------------- ------------------
c39b7e5d-aeca-7a0d-1ddf-005056016fbb be9b7e5d-8fb8-ea92-ba64-005056016fbb 10 40.00 GB 0.01 GB vSAN Default Storage Policy 3 of 3
be9b7e5d-8fb8-ea92-ba64-005056016fbb be9b7e5d-8fb8-ea92-ba64-005056016fbb 10 255.00 GB 0.68 GB vSAN Default Storage Policy 3 of 3
cbadc85b-9500-83ae-3be4-005056016fc3 cbadc85b-9500-83ae-3be4-005056016fc3 7 255.00 GB 1.36 GB vSAN Default Storage Policy 3 of 3
esxcli vsan debug object list|grep -E 'Object UUID:|Version:|Path:'|grep -ve objectVersion:
Object UUID: c39b7e5d-aeca-7a0d-1ddf-005056016fbb
Version: 10
Path: /vmfs/volumes/vsan:524cb6a621a5e774-f81d714149fc60df/be9b7e5d-8fb8-ea92-ba64-005056016fbb/.VM1.vmdk (Exists)
Object UUID: be9b7e5d-8fb8-ea92-ba64-005056016fbb
Version: 10
Path: /vmfs/volumes/vsan:524cb6a621a5e774-f81d714149fc60df/.VM1 (Exists)
Object UUID: cbadc85b-9500-83ae-3be4-005056016fc3
Version: 7
Path: /vmfs/volumes/vsan:524cb6a621a5e774-f81d714149fc60df/.vsan.stats (Exists)
Note: The preceding command excerpts are only examples. Environmental variables may vary depending on the customer's environment.
Environment
VMware vSAN 6.7.x
VMware vSAN 7.0.x
VMware vSAN 7.0.x
Resolution
Option 1:
1) Change the storage policy by doing the following:
a) Click on the cluster name in the left hand pane
b) select Configure>vSAN Services
c) Click on Edit to the left of Performance Service
d) Select a different storage policy from the drop down list
e) From RVC drilled down to the cluster level run vsan.ondisk_upgrade to start a manual ondisk upgrade
2) Run esxcli vsan debug object overview to confirm all objects are on version 10
3) If successful and the original Storage policy for Performance Service is desired, set it back
Note: Changing the storage policy can work for vmdks/namespaces as well that may have failed to upgrade to version 10
Option 2:
If option 1 doesn't resolve the issue try the following:
Disable Performance Service and then re-enable, this will create a new .vsan.stats namespace with version 10
To disable vSAN Performance
1) Select the vSAN cluster
2) Select Configure Tab
3) Under vSAN select Services
4) Select Performance Service and click Edit
5) Click the toggle switch to disable performance service and click Apply
To enable vSAN Performance
1) Since you should already still be at vSAN Services
2) Click Edit to the right of Performance Service
3) Click the toggle switch to enable performance service and click Apply
Note: This will delete any perf stats already collected. Make sure to mention this to the customer in case they are in the middle of troubleshooting a performance issue and need to save the data.
1) Change the storage policy by doing the following:
a) Click on the cluster name in the left hand pane
b) select Configure>vSAN Services
c) Click on Edit to the left of Performance Service
d) Select a different storage policy from the drop down list
e) From RVC drilled down to the cluster level run vsan.ondisk_upgrade to start a manual ondisk upgrade
2) Run esxcli vsan debug object overview to confirm all objects are on version 10
3) If successful and the original Storage policy for Performance Service is desired, set it back
Note: Changing the storage policy can work for vmdks/namespaces as well that may have failed to upgrade to version 10
Option 2:
If option 1 doesn't resolve the issue try the following:
Disable Performance Service and then re-enable, this will create a new .vsan.stats namespace with version 10
To disable vSAN Performance
1) Select the vSAN cluster
2) Select Configure Tab
3) Under vSAN select Services
4) Select Performance Service and click Edit
5) Click the toggle switch to disable performance service and click Apply
To enable vSAN Performance
1) Since you should already still be at vSAN Services
2) Click Edit to the right of Performance Service
3) Click the toggle switch to enable performance service and click Apply
Note: This will delete any perf stats already collected. Make sure to mention this to the customer in case they are in the middle of troubleshooting a performance issue and need to save the data.
Feedback
Yes
No
Powered by
