How to Migrate to a new datastore in EPMC PKS environment

Products

• VMware Cloud PKS

Issue/Introduction

To provide instructions for migrating an existing EPMC PKS environment to a new datastore. This includes service vms and existing clusters.

Environment

VMware PKS 1.x

Resolution

Note: If the Kubernetes clusters have Persistent Volumes, you must use Velero with Restic to back those up and restore them (or you can do this manually copying the VMDKs via vCenter and recreating the PVs in the clusters. Moving PVs across Datastores is a manual/disruptive operation with the in-tree VMware cloud provider. The new Cloud Native Storage CSI driver in PKS 1.7+ uses First Class Disks and is easier for migration.

1. Change the datastore used in EPMC on the TKGI Configuration page.
2. Navigate to the bottom of EPMC day 2 page, and click generate manifest → apply configuration.
3. Log in to Opsman and click on the Bosh tile. On the Director Config page, enable “Recreate VMs deployed by the BOSH Director“
4. Navigate to “Review pending changes“ in OpsMan, and enable the errand “Upgrade all clusters errand“ in PKS and then apply changes. You should see the bosh, PKS and Harbor VMs recreated no the new datastore.
5. You can use bosh CLI to change the datastore for existing clusters. You can use the bosh vms command to query the existing kubernetes cluster deployment, and then use bosh recreate -d <deployment-id> to recreate existing clusters. 
6. For the OpsMan VM, you can use traditional vSphere options (clone, storage migration, etc...) to migrate to the new datastore.