Sharp increase in logs from ESXI host found in Aria Operations for Logs after upgrading to vSphere 8.0.3.x versions

Products

• VMware SDDC Manager
• VMware vSphere ESXi
• VMware vSAN 8.x

Issue/Introduction

After upgrading a VCF environment from 4.x to 5.x version/builds, Aria Operations for Logs (or a third-party log monitoring tool like Splunk) shows a drastic increase in logging on the FDM agent of the primary host in a vSAN cluster with vSphere HA enabled.

Log entries in /var/run/log/fdm.log similar to the following could make up a large portion of the increase:

DsStateChange due to vSAN cluster membership change or access gen no change

changed inventory datastoresInUse changed

Verifying protected state of all vms

Processing protect request for protected vm

Environment

vSphere Cloud Foundation 5.x
vSphere ESXI 8.0.3.x
vSphere vSAN 8.x

Cause

Since 8.0U3, vSAN updates datastore capacity changes in 30 second intervals on each host. The change was to address requests to report datastore usage in near real time manner.

Customers with large vSAN clusters and many VM's actively consuming storage leads to vSAN datastore update events received by FDM very frequently.

Resolution

Engineering is actively looking at potential changes to this logging behavior.  Please open a Broadcom Support case to confirm if you are experiencing this new logging behavior.

Workaround

To suppress this log noise, please ask the customer to follow the steps below:

1. Log in to the vSphere Client (vCenter Web UI).

2. Navigate to Hosts and Clusters.

3. Select the relevant HA-enabled vSAN cluster.

4. Go to Configure > vSphere Availability > Advanced Options.

5. Add the following key-value pairs, the below options are case sensitive:

Option	Value
das.config.level[Cluster].logLevel	info
das.config.level[Cluster].logName	Cluster

6. Click OK.

7. Once the change completes in the Recent Tasks pane, go back to Configure > vSphere Availability.

8. Temporarily disable vSphere HA, wait for completion, then re-enable vSphere HA.

Note: These changes will not reduce the overall FDM logging level to info but will specifically suppress verbose logging for the Cluster component. This ensures we still retain sufficient logs for future troubleshooting.