ESXi ramdisk full due to /var/log/hpHelper.log file size

Products

• VMware vSphere ESXi

Issue/Introduction

When using HP Gen8 hardware, you experience these symptoms:

• You are unable to power on a virtual machine.
• In the /var/log/vobd.log file, you see the entry:
  
  Cannot extend visorfs file /var/log/hpHelper.log because its ramdisk (root) is full
  
  
• The /var/log/hpHelper.log file size is increased.
• Opening a console to a virtual machine fails.
• You see the error:
  
  Unable to connect to the MKS: A general system error occurred: Internal error

Environment

VMware vSphere ESXi 6.0
VMware vSphere ESXi 5.1
VMware vSphere ESXi 5.5
VMware vSphere ESXi 5.0
VMware ESXi 4.1.x Installable

Resolution

To resolve this issue, HP Support has provided an updated hp-ams VIB to stop the excessive logging to the hpHelper.log file.

For more information, see HP Proliant Gen8 Servers - ESXi 5: The /var/log/hpHelper.log File Can Grow Very Large and Fill the ESXi 5 RAMDisk.

You can download ESXi offline bundles from HP Support. For more information, see the HP Support Center.

Alternatively, remove the hpHelper.log file. This prevents the ILO4 management agent reporting on VMkernel or operating system health.

Caution: Remove the hpHelper.log file only if it is not in use.

To remove the hpHelper.log file:

1. Log in to the ESXi host as the root user.
   
   
2. Stop the HP Helper management agent by running the command:
   
   /etc/init.d/hp-ams.sh stop
   
   
3. To remove the hpHelper.log file, run the command:
   
   rm /var/log/hpHelper.log
   
   
4. To restart the HP Helper management agent, run this command (optional):
   
   /etc/init.d/hp-ams.sh start
   
   Note: If the updated hp-ams VIB is not applied, restarting the HP Helper management agent may cause the symptoms to reappear.
   
   
5. In the event of a host reboot, run the command:
   
   chkconfig hp-ams.sh off

This command prevents the hp-ams service from starting after a server reboot.

 

Alternatively, reboot the ESXi host to clear the log files.

Additional Information

For more information on other causes of a full ramdisk, see:

• /var/log/wtmp grows rapidly and fills the ramdisk of the ESXi host (2019021)
• ESXi host is disconnected from vCenter Server due to sfcbd exhausting inodes (2037798)

The /var/log/wtmp file grows rapidly and fills the ramdisk of the ESXi host
ESXi 5.x host is disconnected from vCenter Server due to sfcbd exhausting inodes