Compliant vSAN virtual objects show non-compliant against policy after a VCF upgrade using Cloud Builder

Products

• VMware vSAN

Issue/Introduction

After deploying a cluster using VMware Cloud Foundation (VCF) Cloud Builder, some vSAN objects on vSAN Express Storage Architecture (ESA) may appear as non-compliant with the assigned storage policy

Additionally, the output of the esxcli vsan debug disk summary get command is inconsistent across cluster node

The following error is logged at /var/log/vsanmgmt.log

2025-04-06T07:19:19.101Z Er(11)[+] vsand[2657755] message = 'Timed out when installing CMMDS subscription'

The vsansystem.log file reports incompliance on objects that are expected to be compliant

2025-04-04T19:57:57.374Z In(166) vsansystem[2657773]: [vSAN@6876 sub=Libs opId=e340e9e6-3e73] Object XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX is not compliant, reason: 0x5, incomplainceFlags = 160

Environment

VMware vSAN 8.x

Cause

The issue originates from an internal behavior within vSAN, where the system changes the storage architecture personality type.

vSAN re-identifies the deployment as using the Original Storage Architecture (OSA) instead of the expected Express Storage Architecture (ESA), resulting in storage policy non-compliance.

This behavior is observable in the vmkernel logs, where the value of "VsanPersonalityConfigured" transitions from 1 (OSA) to 2 (ESA), as shown in the following log entries:

2025-04-04T12:16:50.779Z In(182) vmkernel: cpu42:2657772 opID=e384f317)Config: 725: "VsanPersonalityConfigured" = 1, Old Value: 0, (Status: 0x0)

2025-04-04T12:17:49.650Z In(182) vmkernel: cpu52:2658132 opID=62cc4c6b)Config: 725: "VsanPersonalityConfigured" = 2, Old Value: 0, (Status: 0x0)

Resolution

Engineering is aware of this issue when deploying VCF deployments and are planning a fix for a future release

Workaround

• Reactive Workaround: If the issue occurs, rebooting the affected host or restarting the vsanmgmtd service will workaround the issue

 /etc/init.d/vsanmgmtd restart

• Proactive Workaround: As an alternative, in future deployments, you can start vSAN (in this case, vSAN ESA) on all hosts prior to initiating the VCF (VMware Cloud Foundation) workflow. This ensures that vSAN does not need to be enabled during the workflow itself, thereby avoiding the issue altogether.