How to clean up old/ installed bundles in SDDC manager using Bundle Cleanup Utility

Products

• VMware Cloud Foundation
• VMware SDDC Manager / VCF Installer

Issue/Introduction

This article provides instructions for removing old upgrade and install bundles to free up space on the NFS partition on the SDDC Manager. Below are some of the errors that might be encountered when the SDDC Manager NFS partition that stores patch/upgrade bundles runs out of space: 

• Bundle Management Error :  LcmException: LCM crashed during bundle download in SDDC Manager under Bundle Management.

• Attempting to download patch/upgrade bundles fail with "IOException: No space left on device" similar to below:
  
  

 

Environment

VMware Cloud Foundation 3.x
VMware Cloud Foundation 4.x
VMware Cloud Foundation 5.x
VMware Cloud Foundation 9.x

Resolution

Based on the version of SDDC Manager ,refer to the following steps to remove the bundle(s) to free up space:

SDDC 9.x:

1. Navigate to SDDC Manager UI > Lifecycle Management > Binary Management 
2. Review the UPGRADE BINARIES, PATCH BINARIES , INSTALL BINARIES tabs to select the older binaries which are not required
3. Click on the check box beside the component for the binaries intended for removal and use DELETE DOWNLOAD 

SDDC 4.x,5.x: 

1. Browse to bundle management section in the SDDC-Manager UI under Lifecycle management.
2. Refine search to "Downloaded".
   Note : If the bundle has been used to update/upgrade a component, it will be available under "Download history"
3. Open the older bundle that is to be deleted to get the bundle id for step 6
4. Create a snapshot of the SDDC Manager VM
5. SSH to the SDDC Manager as vcf user and elevate to the root user: su
6. From the command line, issue the following command using the bundle id retrieved in step 3

python /opt/vmware/vcf/lcm/lcm-app/bin/bundle_cleanup.py <bundle id>

Notes:

• In case the UI method to delete binaries fails in SDDC 9.x, use the bundle_cleanup.py script as mentioned on kb Resolving "Failed" SDDC Manager Bundle Deletion Errors Following Upgrade 
• In case the bundle removed is a Failed bundle, to release space additional old bundles need to be removed following the similar process 
• For VCF 4.4 and newer versions, the scripted method as per PowerShell Script for VMware Cloud Foundation Bundle Management can also be used
• <bundle id> should be of the form ########-####-####-####-#####
• To find the Bundle ID information for VCF 4.x, 5.x , refer VMware Cloud Foundation - Upgrade Bundle Details

Additional Information

Impact/Risks:

• Bundles cleaned up using this utility will need to be download again in case required for upgrades.
• Upgrade history will be removed after removing bundle from SDDC DB
• LcmException: LCM crashed during bundle download .This issue occurred because bundles in Failed and Pending state were there in SDDC DB. This can be validated from postgres-db-dump.