NSX Edge kni_single Process High CPU Utilization Resulting in Packet Loss

Products

VMware NSX

Issue/Introduction

Multiple users report connectivity issues to database applications.
Packet loss is observed in network traces for multiple applications.
NSX Edge node CPU utilization for the kni_single process remains in the high 90% to 100%.

Within an NSX Edge log bundle, CPU utilization per minute by the kni_single process can be found within this log file: /var/log/vmware/top-cpu.log Example:
#grep 'UTC\|kni_single' /var/log/vmware/top-cpu.log | less

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ TGID COMMAND Mon May 04 12:48:02 UTC 2026 11554 root 20 0 0 0 0 R 3.0 0.0 5334:02 11554 [kni_single] Mon May 04 12:49:01 UTC 2026 11554 root 20 0 0 0 0 S 3.9 0.0 5334:04 11554 [kni_single] Mon May 04 12:50:01 UTC 2026 11554 root 20 0 0 0 0 R 97.1 0.0 5334:54 11554 [kni_single] Mon May 04 12:51:01 UTC 2026 11554 root 20 0 0 0 0 R 98.0 0.0 5335:52 11554 [kni_single]

Observe the sharp increase in CPU utilization by the kni_single process from 3.9% to 97.1% between 12:49 and 12:50.

Tier-0/Tier-1 Edge nodes show persistent high CPU on management threads despite low overall dataplane traffic.
Checksum errors may be reported by the vmnic interfaces on the underlying ESXi hosts.

Environment

VMware NSX 4.2.3.1

Cause

The kni_single kernel thread handles communication between userspace and kernel space. This thread runs on a datapath fastpath core (typically core 0 or 1). When this thread saturates the CPU, it results in packet drops for traffic hashed to those specific cores. Common triggers include:

Concurrent Packet Captures: Active or orphaned capture sessions (span interfaces) drive excessive kni_single utilization.
Driver/Firmware Incompatibility: Some combinations of driver/firmware can result in significant amounts of checksum errors, leading to processing overhead and packet loss.
High Volume L7 Load Balancing: When L7 traffic hits the Edge node, the DPDK fastpath processing engine punts the traffic through the KNI to the Nginx process.
Intensive Firewall/Audit Logging: If "Logging" is enabled on high-hit-count Distributed Firewall (DFW) or Gateway Firewall rules, the Edge must generate and punt these logs to the syslog daemon via the KNI.

Resolution

To check for and delete active or orphaned capture sessions:

1. Log in to the NSX Edge CLI as admin.
2. Check for active packet capture sessions:
  >get capture session
3. Delete any active or unnecessary sessions to clear the span interface:
  >del capture session <session_ID>

To verify the CPU affinity for the kni_single process and then reconfigure it to set its affinity to the non-dataplane cores:

1. Switch to the root user:
  >st en
2. List the available CPU cores:
  #lscpu
  Look for the value after "On-line CPU(s) list:"
3. Check which of the CPU cores are used by the dataplane.
  - Log in as admin user:
    >get dataplane
    Look for the value after "Corelist". These are the cores used for dataplane (DPDK). The cores not listed are the non dataplane cores.
4. Log back in as root user:
  >st en
5. Get the PID of the kni_single kernel thread:
  #ps -aux|grep -i [k]ni_single
6. Use the "taskset" command to list the current CPU affinity for the kni_single process using the PID collected with
  the previous command:
  #taskset -pc <kni_single_PID>
7. If the CPU affinity of kni_single is in fact set to a datapath core, use the below taskset command to reset the CPU affinity to non-datapath cores:
  #taskset -pc 6-7 <kni_single_PID>
  In the above example, cores 0-5 were used for dataplane (DPDK), so the kni_single process was reconfigured for affinity with the two non-dataplane cores, 6 and 7.
8. Verify that the change was made:
  #taskset -pc <kni_single_PID>

Upgrade the vmnic driver and firmware to the latest versions recommended by the hardware vendor to resolve false-positive checksum errors. Finding ESXi Drivers in the Broadcom Support Portal