How to migrate from one s3 endpoint to another s3 endpoint
search cancel

How to migrate from one s3 endpoint to another s3 endpoint

book

Article ID: 297981

calendar_today

Updated On:

Products

VMware Tanzu Application Service for VMs

Issue/Introduction

When using AWS s3 for Bosh Blobstore and/or Tanzu Application Service (TAS) File Storage, what steps should be followed to migrate from an existing s3 bucket into a new one?

Environment

Product Version: 2.6

Resolution

s3 Blobstore Migration

1. Make sure the AWS user has read and write access to both origin and new s3 buckets 
aws s3 ls s3://S3_BUCKET_NAME
aws s3 cp file.txt s3://S3_BUCKET_NAME
aws s3 rm s3://S3_BUCKET_NAME/file.txt
2. Copy the contents of the origin s3 bucket into the new one: 
 aws s3 sync s3://<source_bucket_name> s3://<new_bucket_name>
 --source-region <source_region_name> --region <new_region_name>
3. Change the Blobstore Location on Director Config by manually editing operations manager installation.yml file:
  • ssh into Ops Man VM and backup the existing installation.yml file.
  • Decrypt the installation.yml file. Make sure to use the Ops Manager passphrase when prompted:
sudo -u tempest-web SECRET_KEY_BASE="s" RAILS_ENV=production /home/tempest-web/tempest/web/scripts/decrypt /var/tempest/workspaces/default/installation.yml /tmp/installation.yml
  • Edit the file /tmp/installation.yml and change the values under s3_blobstore_options to the corresponding ones.
endpoint:
bucket_name: 
access_key:
secret_key:
signature_version: 
region:
  • Encrypt installation.yml again:
sudo -u tempest-web RAILS_ENV=production
/home/tempest-web/tempest/web/scripts/encrypt /tmp/installation.yml
/var/tempest/workspaces/default/installation.yml
4. Reload Director Config in Ops Manager UI, it should display the newly configured values.



5. Run Apply Changes for BOSH Director.

Verify your changes and the s3_blobstore configuration file in the BOSH VM and verify that it has been updated with the new s3 bucket information: 
find /var -type f -iname '*s3_blobstore_config*'
cat s3_blobstore_config-8bf6e345-ea00-4d93-b04f-e64dff904ccf | grep bucket_name

PAS s3 migration

1. Make sure again that the AWS user has read and write access to both origin and new s3 buckets. Also that there are no apps getting pushed while the procedure takes place.

2. Copy the contents of the origin s3 buckets into the new ones: 
 aws s3 sync s3://<source_bucket_name> s3://<new_bucket_name>
 --source-region <source_region_name> --region <new_region_name>

3. Edit the TAS Tile File Storage s3 filestore to the new location:



Verify: Make sure the Recreate All VMs checkbox is selected in Director Config on the BOSH Director Tile:


5. Run Apply Changes: make sure you also select the Service Tiles check boxes.

Verify your changes and Check that bucket_name has changed to the new s3 bucket location on the blobstore-s3.json configuration file: 
bosh -d CF_DEPLOYMENT_NAME ssh -c "sudo grep bucket_name /var/vcap/bosh/etc/blobstore-s3.json"

Note: Running Apply changes with Recreate All VMs checkbox enabled won't recreate other service instances VMs. These will need to be manually restarted for the blobstore-s3.json configuration files to be updated.

In order to manually recreate the VMs, you can run the following: 
bosh -d service-instance_GUID recreate


Powered by Wolken Software