Getting "Catalog health is skipped: unhealthy invalidArgument" warnings on hostd logs of the ESXi Host.

Products

• VMware vCenter Server

Issue/Introduction

• The below events are seen in the host-catalog.log file or hostd logs of the ESXi Host.

             Init:277    Initializing catalog
             Init:153    1 vclock file(s) found<1>: [vclock-11]
             SetHealthStatus:3187    Catalog health is set: healthy 1
             SetHealthStatus:3179    Catalog health is skipped: unhealthy invalidArgument 21

• However, on checking the FCD database, all the disks are present, the vClock and tidy information is correct and you're able to confirm the Persistent Volume to FCD mapping.
  
  
• Additionally, there are no issues with the volume attachment de-attachment operations so pods requiring persistent storage are able to correctly mount and unmount volumes if and when required.

Environment

VMware vCenter Server
VMware vSphere ESXi
vSphere CSI Driver

Cause

The message - "Catalog health is skipped: unhealthy invalidArgument" is a harmless message and is not an error. When multiple hosts migrate tidy at the same moment, the first host will succeed, while other hosts stop with invalidArgument. This invalidArgument does not indicate any unhealthy status, it just echoes the "invalidArgument" log and skips it.

Resolution

This message/warning can be safely ignored, provided that the FCD catalog of the datastores is in sync and there are no issues related to volume attachment/de-attachment because of the sync issues between CNS and CSI controller.