Improper termination of pktcap-uw causes Userworld exhaustion and ESXi host management agent failures

Products

• VMware vSphere ESXi

Issue/Introduction

The purpose of this article is to raise awareness about this issue and inform you of the proper methods to terminate pktcap-uw.


Symptoms:

When using the enhanced packet capture tool pktcap-uw on an ESXi 5.5 or 6.0 host, you experience these symptoms:

• The ESXi host is disconnected from vCenter Server.
• You see entries in the /var/log/vmkernel.log file, similar to:
  
  2015-08-25T16:25:58.340Z cpu25:35188)WARNING: World: 2361: Max worlds exceeded.
  2015-08-25T16:25:59.024Z cpu8:34944)WARNING: World: 2361: Max worlds exceeded.
  
  Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.
  
  
• Any active ESXi Shell, or ssh session returns this error when attempting to execute any command:
  
  -sh: can't fork
  
  

Environment

VMware vSphere ESXi 5.5
VMware vSphere ESXi 6.0

Cause

If pktcap-uw was terminated with any signal other than SIGINT or -2 (numeric value for SIGINT) only the userworld frontend is terminated leaving the hypervisor process to spawn userworlds until depletion.

Resolution

This is an expected behavior when pktcap-uw is terminated incorrectly.

To resolve this issue, reboot the ESXi host.

To terminate the pktcap-uw command correctly:

• In ESXi 5.5, you must use a CTRL^C, kill -2, or kill -SIGINT command.
• In ESXi 6.0, you can use any of the commands listed above or just the kill command. In ESXi 6.0, the kill command defaults to kill -2 automatically.

If the pktcap-uw command is terminated by any other signal, reboot the host as soon as possible.

Additional Information

For more information about the pktcap-uw tool, see Using the pktcap-uw tool in ESXi 5.5 and later.