Transaction Traces partitions are getting filled 100% in few days

Products

• CA Application Performance Management Agent (APM / Wily / Introscope)
• CA Application Performance Management (APM / Wily / Introscope)
• INTROSCOPE
• DX Application Performance Management

Issue/Introduction

We are observing issues on APM collectors where transaction traces partitions are getting filled 100% in few days.

We ran the IndexRebuilder.sh script few days back, but we see that partition /data/traces is 100% full again.

Environment

Release : 10.7.0

Component : Introscope

Cause

It seems like this may be some issue with the file system itself on the server.

We see the following on the sever.

df -h on /data/traces file system shows 48GB used.

du -h on /data/traces traces directory shows 35GB used. 

Resolution

It seems like issue is due to some issue with the file system on the server. Rebooting the server corrected the issue with space on the file system.