After upgrading vCenter to 8.0U3 vSAN Skyline Health triggers the alert "Cluster Configuration Consistency"

Products

• VMware vSAN

Issue/Introduction

• vSAN Cluster running version 7.x or 8.x
• vSAN Encryption is enabled
• A shallow rekey was performed on the cluster while vCenter was on version 7.x or 8.x below 8.0U3
• vCenter upgraded to 8.U3 or above the vSAN Skyline Health triggers the alert "Cluster Configuration Consistency" - "The host (cluster) has completed encryption configuration, but the cluster (host) has not completed."
• There is no active tasks running on vCenter with regards to encryption for the vSAN cluster.
• In vCenter health logs we see the following: 

  • /var/log/vmware/vsan-health/vmware-vsan-health-service.log reports that the vCenter encryption "changing" flag is "None" but the host encryption "changing" flag is "False":

    2024-08-05T09:55:30.769+09:00 WARNING vsan-mgmt[2679179] [VsanHealthEncUtil::_AggregateEncryptionConfigHealth opID=b4e991d2] Host: host_name has inconsistent config with cluster, field: changing, vc value: None, host value: False

  • /var/log/vmware/vsan-health/vmware-vsan-health-summary-result.log:
    2024-08-05T09:55:41.902+09:00 INFO vsan-mgmt[2679179] [VsanHealthSummaryLogUtil::PrintHealthResult opID=b4e991d2] Cluster vSAN_Cluster_name  Overall Health : red
   Group cluster health : red
      Test timedrift health : green
      Test advcfgsync health : green
      Test clomdliveness health : green
      Test resynclimit health : green
      Test vsandconfigconsistency health : green
      Test vcauthoritative health : green
      Test consistentconfig health : red
         Issues: Host  Disk  Issue  Recommendation
                 (Host-266, '', Host(Cluster)HasFinishedEncryptionConfigurationButCluster(Host)HasNot., Click'RemediateInconsistentConfiguration'), (Host-251, '', Host(Cluster)
HasFinishedEncryptionConfigurationButCluster(Host)HasNot., Click'RemediateInconsistentConfiguration'),
                 (Host-120, '', Host(Cluster)HasFinishedEncryptionConfigurationButCluster(Host)HasNot., Click'RemediateInconsistentConfiguration'), (Host-236, '', Host(Cluster)
HasFinishedEncryptionConfigurationButCluster(Host)HasNot., Click'RemediateInconsistentConfiguration'),
                 (Host-235, '', Host(Cluster)HasFinishedEncryptionConfigurationButCluster(Host)HasNot., Click'RemediateInconsistentConfiguration'), (Host-243, '', Host(Cluster)
HasFinishedEncryptionConfigurationButCluster(Host)HasNot., Click'RemediateInconsistentConfiguration'),

Environment

VMware vCenter Server 8.0U3

VMware vSAN

 

Cause

In vCenter below 8.0U3, when a shallow rekey is performed on a vSAN encrypted cluster, vCenter records the result value as None and the host value is recorded as False, as of version 8.0U3 vCenter records the result as False to match the host value.

Since the shallow rekey was performed while vCenter was on version below 8.0U3 and then vCenter upgraded to 8.0U3 the None value carries over resulting in a false positive. 

Resolution

Clicking on the "Remediate inconsistent configuration" will NOT resolve this issue nor will upgrading the cluster to 8.0U3.

vSAN engineering is aware of this issue and is due to be fixed in a future vCenter version, for now you can just silence this alert as this is just cosmetic and there is no impact to the cluster.