When using VEEAM One version 10/10a, vSAN tasks may get hung or fail and hosts may report as not responding.

Products

• VMware vSAN

Issue/Introduction

Symptoms:
When using VEEAM One version 10/10a, vSAN tasks may get hung or fail and hosts may report as not responding.

Environment

VMware vSAN 6.7.x
VMware vSAN 7.0.x

Cause

VEEAM ONE software collects vSAN statistics by performing a large number of vSAN internal API calls to queryVsanStatistics() and queryCmmds() which leads to outages in the vSAN management stack.

In vsansystem you see:
2021-04-28T18:08:09.141Z info vsansystem[2270704] [vSAN@6876 sub=AdapterServer opId=57ba76b-ad-406e-1536] Invoking 'queryCmmds' on 'ha-vsan-internal-system', session '########-####-####-####-########b671'
2021-04-28T18:08:09.141Z info vsansystem[2270683] [vSAN@6876 sub=AdapterServer opId=792c649c-54-406d-1537] Invoking 'queryVsanStatistics' on 'ha-vsan-internal-system', session '########-####-####-####-########b671'
2021-04-28T18:08:29.151Z info vsansystem[2270694] [vSAN@6876 sub=AdapterServer opId=68645f28-62-4201-1572] Invoking 'queryVsanStatistics' on 'ha-vsan-internal-system', session '########-####-####-####-########b671'
2021-04-28T18:08:29.167Z info vsansystem[2270708] [vSAN@6876 sub=AdapterServer opId=4d3a0bc9-f4-4202-1573] Invoking 'queryCmmds' on 'ha-vsan-internal-system', session '########-####-####-####-########b671'

In hostd you see:
2021-04-28T18:08:09.141Z info vsansystem[2270704] [vSAN@6876 sub=AdapterServer opId=57ba76b-ad-406e-1536] Invoking 'queryCmmds' on 'ha-vsan-internal-system', session '########-####-####-####-########b671'
2021-04-28T18:08:09.141Z info vsansystem[2270683] [vSAN@6876 sub=AdapterServer opId=792c649c-54-406d-1537] Invoking 'queryVsanStatistics' on 'ha-vsan-internal-system', session '########-####-####-####-########b671'
2021-04-28T18:08:29.151Z info vsansystem[2270694] [vSAN@6876 sub=AdapterServer opId=68645f28-62-4201-1572] Invoking 'queryVsanStatistics' on 'ha-vsan-internal-system', session '########-####-####-####-########b671'
2021-04-28T18:08:29.167Z info vsansystem[2270708] [vSAN@6876 sub=AdapterServer opId=4d3a0bc9-f4-4202-1573] Invoking 'queryCmmds' on 'ha-vsan-internal-system', session '########-####-####-####-########b671'

Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.

Resolution

Upgrade VEEAM One to version 11 as per VEEAM KB https://www.veeam.com/kb4234.

Additional Information

Impact/Risks:
vSAN tasks my hang/fail and hosts may report as not responding.