High RDT Checksum Mismatch count detected

Products

• VMware vSAN

Issue/Introduction

Symptoms:

• ESXi/vCenter 8.0 or higher
• High RDT Checksum Mismatch count alerts
• You see the below messages in vmkernel and vmkwarning logs, where the header IP mentioned is a network scanner

2023-09-18T17:14:41.185Z Wa(180) vmkwarning: cpu40:2116951)WARNING: RDTTCPConn: RDTTCPReceive:3539: 0x43204475ab40(0x0): unexpected RDT >header from IP 10.xx.xx.40 to IP 192.xxx.xx.15 (vmknic vmk0): version 22 type 3 length 454 optionLen 1 ctr: 45
2023-09-18T18:03:59.206Z Wa(180) vmkwarning: cpu39:2116951)WARNING: RDTTCPConn: RDTTCPReceive:3539: 0x43204475ab40(0x0): unexpected RDT >header from IP 10.xx.xx.40 to IP 192.xxx.xx.15 (vmknic vmk0): version 22 type 3 length 454 optionLen 1 ctr: 54
2023-09-24T21:31:45.506Z Wa(180) vmkwarning: cpu39:2116951)WARNING: RDTTCPConn: RDTTCPReceive:3539: 0x43204475ab40(0x0): unexpected RDT >header from IP 10.xx.xx.40 to IP 192.xxx.xx.15 (vmknic vmk0): version 22 type 3 length 454 optionLen 1 ctr: 63

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Environment

VMware vSAN 8.0.x

Cause

A new health check "checksumMismatchCount" was introduced in vSAN 8.x which may be triggered when RDT TCP port 2233 receives non-RDT traffic.

The threshold is:
- yellow: 1 checksumMismatchCount in 5 minutes
- red: 10 checksumMismatchCount in 5 minutes

If network scanners are in use and polling on RDT port 2233, this introduces non-RDT traffic and thus triggering this alert.

Resolution

• Remove RDT port 2233 from the network scanner tasks.
• If there are no network scanners in the environment open a case with vSAN support for further investigation.

Additional Information

VMware Ports and Protocols: vSAN
vSAN Networking Terms and Definitions