Objects may show non-compliant against policy when they are actually compliant.

Products

• VMware vSAN

Issue/Introduction

Symptoms:

vSAN objects may show as non-compliant with storage policy when hosts are moved between vSAN OSA cluster and ESA cluster.

When changing the OSA cluster to an ESA cluster, few hosts will encounter objects that are non-compliant on the ESA cluster when other hosts show them as compliant.

This situation occurs when the user wants ESA cluster to be used instead of OSA. A new cluster was created and vSAN ESA was enabled on it. Later, the hosts were removed from the OSA enabled cluster and moved to the ESA enabled cluster.

The user repurposed a host from an OSA cluster and added it to an ESA cluster without properly decommissioning the host from the OSA cluster first.

Environment

VMware vSAN 8.0.x

Cause

This issue is observed when the host was not properly decommissioned from the OSA cluster.

As a result, the host carried over the OSA version and personality settings, which causes the conflict.

Below log snippets explains the difference between the versions.

vsansystem.log:
vsansystem.0:2023-11-10T21:13:15.487Z In(166) vsansystem[1000345036]: [vSAN@6876 sub=Libs opId=41b29d8b-60-03-77-ecab-1934] VsanInfoImpl: Joining vSAN cluster 52b6ceda-848a-e13a-ef5e-xxxxxxxx with boot:0, enableVsan2:0
vsansystem.0:2023-11-10T21:13:54.782Z In(166) vsansystem[1000345036]: [vSAN@6876 sub=CMMDSUtil opId=41b29d8b-60-03-77-ecab-1934] Initialized VSAN Compliance library.
vsansystem.0:2023-11-10T21:14:24.086Z In(166) vsansystem[1000346244]: [vSAN@6876 sub=VsanSystemProvider opId=41b29f81-53-03-ba-edc9-19e4] Leaving cluster UUID: 52b6ceda-848a-e13a-ef5e-xxxxxxxx
vsansystem.0:2023-11-10T21:14:45.310Z In(166) vsansystem[1000346244]: [vSAN@6876 sub=Libs opId=41b29f81-53-03-ba-edc9-19e4] VsanInfoImpl: Joining vSAN cluster 52b6ceda-848a-e13a-ef5e-xxxxxxxx with boot:0, enableVsan2:1

vmkernel.log:
vmkernel.log: 2023-11-10T21:13:15.501Z In(182) vmkernel: cpu0:1000345036 opID=9cda2603)Config: WriteIntegerInternalCommon:791: "VsanPersonalityConfigured" = 1, Old Value: 0, (Status: 0x0)
vmkernel.log: 2023-11-10T21:14:45.274Z In(182) vmkernel: cpu0:1000346244 opID=b8077d55)Config: WriteIntegerInternalCommon:791: "VsanPersonalityConfigured" = 0, Old Value: 1, (Status: 0x0)
vmkernel.log: 2023-11-10T21:14:45.320Z In(182) vmkernel: cpu0:1000346244 opID=b8077d55)Config: WriteIntegerInternalCommon:791: "VsanPersonalityConfigured" = 2, Old Value: 0, (Status: 0x0)

OSA
enableVsan2:0
"VsanPersonalityConfigured" = 1,

ESA
enableVsan2:1
"VsanPersonalityConfigured" = 2,

Resolution

Ensure that you properly decommission an OSA host before  adding it to an ESA enabled cluster.

Refer the below document for more information on decommissioning an ESXi host from cluster.

Permanently Decommissioning a node from a vSAN Cluster (2148975)

Workaround:

Place the host that report the objects as non-compliant in maintenance mode and reboot the hosts. The document below explains about putting a host into maintenance mode.

Place a Member of vSAN Cluster in Maintenance Mode

Additional Information