High Frequency Execution of esxcli Commands on ESXi
search cancel

High Frequency Execution of esxcli Commands on ESXi

book

Article ID: 419983

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

The command "esxcli software vib get" is being executed repeatedly on the ESXi host, typically at intervals of approximately three seconds.

The following corresponding entries can be observed in the hostd.log and envoy-access.log files:

  • hostd.log
    YYYY-MM-DDTHH:mm;ss.SSSZ In(166) Hostd[525475]: [Originator@6876 sub=Solo.VmwareCLI opID=esxcli-5f-74ae sid=52823c4a user=root] Dispatch software.vib.get
    YYYY-MM-DDTHH:mm;ss.SSSZ In(166) Hostd[525475]: [Originator@6876 sub=Solo.VmwareCLI opID=esxcli-5f-74ae sid=52823c4a user=root] Dispatch software.vib.get done
    YYYY-MM-DDTHH:mm;ss.SSSZ In(166) Hostd[525460]: [Originator@6876 sub=Solo.VmwareCLI opID=esxcli-cc-74bc sid=5226ae87 user=root] Dispatch software.vib.get
    YYYY-MM-DDTHH:mm;ss.SSSZ In(166) Hostd[525460]: [Originator@6876 sub=Solo.VmwareCLI opID=esxcli-cc-74bc sid=5226ae87 user=root] Dispatch software.vib.get done
    YYYY-MM-DDTHH:mm;ss.SSSZ In(166) Hostd[525482]: [Originator@6876 sub=Solo.VmwareCLI opID=esxcli-60-74ca sid=520ac49e user=root] Dispatch software.vib.get
    YYYY-MM-DDTHH:mm;ss.SSSZ In(166) Hostd[525482]: [Originator@6876 sub=Solo.VmwareCLI opID=esxcli-60-74ca sid=520ac49e user=root] Dispatch software.vib.get done
    YYYY-MM-DDTHH:mm;ss.SSSZ In(166) Hostd[525487]: [Originator@6876 sub=Solo.VmwareCLI opID=esxcli-d4-74db sid=52fe7cc2 user=root] Dispatch software.vib.get
    YYYY-MM-DDTHH:mm;ss.SSSZ In(166) Hostd[525487]: [Originator@6876 sub=Solo.VmwareCLI opID=esxcli-d4-74db sid=52fe7cc2 user=root] Dispatch software.vib.get done
  • envoy-access.log
    YYYY-MM-DDTHH:mm;ss.SSSZ In(166) envoy-access[525161]: POST /sdk/webService HTTP/1.1 200 via_upstream - 571 141213 1350 1350 0 127.0.0.1:24669 - 127.0.0.1:80 127.0.0.1:57388 - 127.0.0.1:8307 "esxcli-5f" "VimEsxCLIsoftwarevibget"
    YYYY-MM-DDTHH:mm;ss.SSSZ In(166) envoy-access[525161]: POST /sdk/webService HTTP/1.1 200 via_upstream - 571 141213 1336 1335 0 127.0.0.1:60512 - 127.0.0.1:80 127.0.0.1:27220 - 127.0.0.1:8307 "esxcli-cc" "VimEsxCLIsoftwarevibget"
    YYYY-MM-DDTHH:mm;ss.SSSZ In(166) envoy-access[525161]: POST /sdk/webService HTTP/1.1 200 via_upstream - 571 141213 1324 1324 0 127.0.0.1:21072 - 127.0.0.1:80 127.0.0.1:50038 - 127.0.0.1:8307 "esxcli-60" "VimEsxCLIsoftwarevibget"
    YYYY-MM-DDTHH:mm;ss.SSSZ In(166) envoy-access[525161]: POST /sdk/webService HTTP/1.1 200 via_upstream - 571 141213 1342 1342 0 127.0.0.1:55875 - 127.0.0.1:80 127.0.0.1:24360 - 127.0.0.1:8307 "esxcli-d4" "VimEsxCLIsoftwarevibget"

Environment

ESXi 8.x

Cause

To identify the specific process responsible for this behavior, the following command was executed to capture process (ps) output 20 times over a 10-second duration:

# i=1; while [ $i -le 20 ]; do echo "--- Snapshot $i/20 ---"; ps -cPTgjstz; sleep 0.5; i=$((i + 1)); done

Reviewing the output reveals that the process /opt/sut/bin/sut is active, along with its child processes:

######2   ######2  sut                                 ######2  ######2  ######2  ######1   U       WAIT    UWAIT   0-31   1730.884117  /opt/sut/bin/sut
#######9  #######9  sh                                  #######9  ######2  ######2  ######2   U       WAIT    UWAIT   0-31      0.1254    sh -c esxcli software vib get -n ilo &>/dev/null
#######0  #######0  python                              #######0  ######2  ######2  #######9  U       WAIT    KTCP    0-31      0.274839  python /sbin/esxcli software vib get -n ilo
#######1  #######1  sh                                  #######1  ######0  ######0  ######0   U       WAIT    UWAIT   0-31      0.0673    /bin/sh -c LANG=en_US.UTF-8 /usr/lib/vmware/esxcli-software vib.get  -n "ilo" 
#######2  #######2  python                              #######2  ######0  ######0  #######1  U       RUN     NONE    0-31      0.103877  /usr/bin/python /usr/lib/vmware/esxcli-software vib.get -n ilo

This indicates that the esxcli command is being triggered by the HPE Smart Update Tool (SUT).

Note: This behavior is caused by third-party software and is not a VMware product issue.

Resolution

This issue must be addressed by the vendor who provides the software.

Contact your server hardware vendor (e.g., HPE) for further details.

Additional Information

The high-frequency execution of esxcli commands may increase the likelihood of resource contention or system load, potentially leading to intermittent occurrences of the event:

"hostd detected to be non-responsive" event occurs intermittently

Powered by Wolken Software