OpsManager upgrade fails on AWS with EBS volume encryption via OM CLI
search cancel

OpsManager upgrade fails on AWS with EBS volume encryption via OM CLI

book

Article ID: 293776

calendar_today

Updated On:

Products

Operations Manager

Issue/Introduction

During Upgrade a new VM is initially created as 8GB and then resized (200 GB default)
If Amazon EBS encrypted volume is enabled, upgrade of OpsManager VM via om-cli stops with below error. 
om vm-lifecycle upgrade-opsman --config opsman.yml --image-file ops-manager-aws-x.x.x.yml 


Executing: "aws ec2 modify-volume --volume-id vol-0252db8d74301c7c9 --size 200"
This could take a few moments...
aws[stderr]:
aws[stderr]: An error occurred (VolumeModificationRateExceeded) when calling the ModifyVolume operation: You've reached the maximum modification rate per volume limit. Wait at least 6 hours between modifications per EBS volume.
volume not available to configure yet, polling in 2m0s
After the failure, manual resize of the OpsManager VM can be done from the AWS UI


Environment

Product Version: 2.10

Resolution

om vm-lifecycle upgrade-opsman does not support encrypting OM EBS Volume using AWS KMS Customer Master Key.
AWS has a rate limit for 6 hours between volume updates. So part of the upgrade process, om code is attempting to modify the volume twice which is causing the failure.

As a workaround, disable EBS Encryption(enabled by default), run the upgrade and re-enable encryption.