EBS Volumes Incorrectly Deleted by Policy Due to Polling Interval Gaps
search cancel

EBS Volumes Incorrectly Deleted by Policy Due to Polling Interval Gaps

book

Article ID: 447192

calendar_today

Updated On:

Products

CloudHealth

Issue/Introduction

A CloudHealth policy configured to delete unattached EBS volumes (e.g., "Unattached for 2 weeks") executes a deletion on volumes that are actually in use.

  • The user reports the volumes are attached briefly at regular intervals (e.g., for 2 minutes every 30 minutes).
  • AWS CloudTrail confirms the DeleteVolume action was initiated by the CloudhealthRole.
  • The CloudHealth Asset Profile shows a continuous "Unattached Date" despite the intermittent attachments.

Environment

  • Platform: CloudHealth
  • Cloud Provider: Amazon Web Services (AWS)
  • Resource Type: EBS Volumes (Elastic Block Store)

Cause

This issue is caused by a timing gap between CloudHealth's background API discovery cycles and the asset's brief attachment windows.

CloudHealth collects infrastructure configuration metadata via automated API snapshots (such as DescribeVolumes) on a standard 24-hour interval cycle.

If a volume is attached for only a few minutes and then detached, it remains in an "Available" (unattached) state for the vast majority of the day. If the daily CloudHealth discovery scan consistently lands during the "Available" window, the platform never records an "Attached" status. Consequently, the internal "Unattach Date" counter is never reset, leading the policy engine to calculate the duration based on a continuous unattached state.

Resolution

Option 1: Exemption Tags (Recommended)

The most effective way to protect these volumes is to exclude them from the policy using metadata tags.

  1. Apply a Tag in AWS: Add a unique tag to the volumes involved in intermittent tasks (e.g., PolicyExclude: True).
  2. Modify the Policy:
    • Navigate to Setup > Governance > Policies.
    • Edit the affected EBS Deletion Policy.
    • Click Add Condition.
    • Select Volume Tag -> PolicyExclude -> Does Not Equal -> True.
    • Save the policy.

Option 2: Change Policy Action

Change the policy action from "Delete" to an Alert "Email".  This introduces a manual verification step, allowing an administrator to check the volumes' specific use case before they are permanently removed.

Option 3: Increase Visibility

Utilize FlexReports or the EBS Volume Hours report to review usage patterns over a broader window, though note that these also rely on the same discovery cycles.