How to clean up temporary Hive folders

Products

• Services Suite

Issue/Introduction

This article tells you how to clean up Hive temporary files and folders to free up the client host disk space.

Environment

Cause

The host is running out of disk space because of the large size of the /tmp folder.

Resolution

This script cleans up data in the /tmp folder that is older than N seconds:

find /tmp -mmin +N -exec rm {} \;

For example, this command will find files in /tmp that are older than 1 hour and remove them:

find /tmp -mmin +3600 -exec rm {} \;

IMPORTANT NOTE: This command will permanently remove files so it should be used with caution.

The command can be put into a cron-job so that it can run regularly on the system.

NOTE: Temporary files may also be created in HDFS in /tmp and they might need to be cleaned up periodically also.