Many current.png.xxxx on the /tmp on ESXi host causes hostd to crash and disconnect

Products

• VMware Cloud Director
• VMware vSphere ESXi

Issue/Introduction

Symptoms:

• VMware ESXi 5.x hosts become disconnected.
• Management Agent hostd crashes.
• In the /var/log/vmkwarning log file of the ESXi host, you see entries similar to:
  
  2012-10-05T12:38:48.924Z cpu18:3858705)WARNING: VisorFSObj: 893: Cannot create file tmp:/current.png5557 for process vmx-mks:Cell Profiler x64 (b9d6aa64-d789-4954-bde9-f2875406c159) because the visorfs inode table is full. 2012-10-05T12:38:49.808Z cpu27:6014573)WARNING: VisorFSObj: 893: Cannot create file /etc/vmware/esx.conf.LOCK for process esxcfg-advcfg because the visorfs inode table is full.
  
  
• In the /var/log/vmware/hostd.log file of the ESXi host, you see the error:
  
  cpu11:916176)WARNING: VisorFSObj: 893: Cannot create file /etc/vmware/esx.conf.LOCK for process hostd because the visorfs inode table is full.

• /tmp on the ESXi host has files similar to:
  
  -rw------- 1 root root 0 Jul 23 14:34 current.png2184
  -rw------- 1 root root 0 Jul 23 14:34 current.png2278
  -rw------- 1 root root 0 Jul 23 14:34 current.png2144
  -rw------- 1 root root 0 Jul 23 14:34 current.png1972
  -rw------- 1 root root 0 Jul 23 14:35 current.png2274
  -rw------- 1 root root 0 Jul 23 14:35 current.png2296
  -rw------- 1 root root 0 Jul 23 14:35 current.png2227
  -rw------- 1 root root 0 Jul 23 14:35 current.png2153
  -rw------- 1 root root 0 Jul 23 14:36 current.png2268
  -rw------- 1 root root 0 Jul 23 14:36 current.png2267
  -rw------- 1 root root 0 Jul 23 14:36 current.png2298
  -rw------- 1 root root 0 Jul 23 14:36 current.png2409

Environment

VMware Cloud Director 1.5.x
VMware vSphere ESXi 5.0

Cause

This error may occur when a datastore gets disconnected and vCloud Director/ESXi cannot clear temporary thumbnails image files that vCloud Director uses to display updated thumbnail views of the virtual machine.

Resolution

To resolve this issue, you should reboot the ESXi host. Ensure there are no Storage or Datastore problems on the host.

Alternatively, you can create a cronjob.

Note: The cronjob does not persist across host reboots.

For example:

/var/spool/cron/crontabs

# cat root

#min hour day mon dow command

1 1 * * * /sbin/tmpwatch.py

1 * * * * /sbin/auto-backup.sh

0 * * * * /usr/lib/vmware/vmksummary/log-heartbeat.py

15 * * * * echo `date` there are `ls /tmp/current.png* | wc -l` files. deleting all current.png files `rm /tmp/current.png*` >> /tmp/delete.log