This article explains how to move a vSAN cluster from one vCenter Server to another vCenter Server.
VMware vSAN 6.x
VMware vSAN 7.0.x
VMware vSAN 8.0.x
The following steps can be followed when migrating vSAN clusters from one vCenter to another vCenter and when the original vCenter is offline permanently or unable to be restored from backup.
Notes:
- When using vSphere Standard Virtual Switch no additional steps are required other than ensuring the physical networking routes are working as designed.
- When using vSphere Distributed Switch, please reference further material/instruction KB links below to prepare the vDS for migration.
Note: While importing vDS settings, do not select preserve original distributed switch port group identifiers as this could negatively impact the migration.
- When using NSX-T NVDS with standalone host configuration, no additional steps are required other than ensuring the physical networking routes are working as designed.
Note: If using Transport Node Profile for NSX-T NVDS, then need to migrate to standalone host configuration.
For more information on the migration of Hosts, see Moving an ESX/ESXi host with vDS from one vCenter Server to another
For more information on migration of vDS see Exporting/importing/restoring Distributed Switch configs using vSphere Web Client
Backing Up and Restoring a vSphere Distributed Switch Configuration
For more information on vSphere see https://docs.vmware.com/
- Once the cluster networking configuration requirements are satisfied, then may proceed with the next steps of the migration.
4. Determine if vSAN Data at Rest Encryption is in use and make certain that keys and the KMS configurations are well known. See the guide for further guidance in preparing the cluster for migration.
Also, See further details in this instruction on step 7a.
Document: Using Encryption on a vSAN Cluster
5. Create a new cluster with DRS, vSphere HA, and EVC Disabled. These options should only be enabled after the cluster is moved and stable on the new vCenter.
6. Create a new cluster Inventory object.
7. a. Enable vSAN along with required services matching the original cluster parameters such as Encryption and or Deduplication and Compression.
- If the original vCenter is not available to compare configurations, the following command can be used to list the vSAN services in use.
localcli vsan storage list |grep -i 'Device\|Is ssd\|Is Capacity Tier\|DEDUPLICATION\|COMPRESSION\|In CMMDS\|ENCRYPTION:' |sed 'N;N;N;N;;N;N;s/\n//g' | sort -k9;
localcli vsan storage list |grep -i 'Device\|Is ssd\|Is Capacity Tier\|DEDUPLICATION\|COMPRESSION\|In CMMDS\|ENCRYPTION:' |sed 'N;N;N;N;;N;N;s/\n//g' | sort -k9;
Device: mpx.vmhba0:C0:T1:L0 Is SSD: true In CMMDS: true Deduplication: false Compression: false Is Capacity Tier: false Encryption: true
Device: mpx.vmhba0:C0:T2:L0 Is SSD: true In CMMDS: true Deduplication: false Compression: false Is Capacity Tier: true Encryption: true
b. When KMS Encryption service is in use -
Make certain to add the KMS servers to the newly built vCenter server with the same KMS-Cluster names and complete all required steps to add the KMS servers into the new vCenter server.
Example:
# esxcli vsan encryption kms list
Cluster Name KMS Address Proxy Address Proxy Port Username
-------------------- ----- ----------------- ------------- ---------- --------
####-KMS-Cluster KMS-1 ##.###.##.##:#### N/A N/A N/A
# esxcli vsan encryption info get
Attribute Value
------------------- -----
kekId ########-####-####-####-############
dekGenerationId 1
enabled True
hostKeyId
########-####-####-####-############
eraseDisksBeforeUse False
changing False
8. Create the Virtual Machine vSAN storage policies that match the vSAN policies from the old cluster. Ensure that rulesets have matching policies. If the VMs are migrated without matching policies, vSAN may need to perform a resync to ensure the VMs and objects are compliant.
If the policy names and policy attributes were unknown due to old vCenter unavailability, the following commands can be used to obtain the policy settings:"esxcli vsan debug object list | grep spbmProfileId | sort | uniq" or "esxcli vsan debug object list | grep spbmProfileName | sort | uniq"
Review the attributes for each object using the policies by running "esxcli vsan debug object list |less"
and search by the policy name (found above).
Example :
# esxcli vsan debug object list | grep spbmProfileId | sort | uniq
spbmProfileId:
########-####-####-####-############
spbmProfileId:
########-####-####-####-############
# esxcli vsan debug object list | grep spbmProfileName | sort | uniq
spbmProfileName: VM Storage Policy-DN
spbmProfileName: vSAN Default Storage Policy
An example is provided below of the policy attributes for the spbmProfileName: vSAN default storage policy
Run Command: esxcli vsan debug object list | less
Example:
Object UUID:
########-####-####-####-############
Version: 10
Health: healthy
Owner: server.com
Size: 15.00 GB
Used: 0.89 GB
Policy: ** CONTENT BELOW INDICATES POLICY ATTRIBUTES**
cacheReservation: 0
forceProvisioning: 0
spbmProfileId:
########-####-####-####-############
proportionalCapacity: 0
spbmProfileGenerationNumber: 0
stripeWidth: 1
spbmProfileName: vSAN Default Storage Policy
CSN: 16
hostFailuresToTolerate: 1
Configuration:
RAID_1
Component:
########-####-####-####-############
Component State: ACTIVE, Address Space(B): 16106127360 (15.00GB), Disk UUID:
########-####-####-####-############
, Disk Name: naa.###############:2 Votes: 1, Capacity Used(B): 486539264 (0.45GB), Physical Capacity Used(B): 478150656 (0.45GB), Host Name:server.com
Component:
########-####-####-####-############
Component State: ACTIVE, Address Space(B): 16106127360 (15.00GB), Disk UUID:
########-####-####-####-############
, Disk Name: naa.###############
:2 Votes: 1, Capacity Used(B): 486539264 (0.45GB), Physical Capacity Used(B): 478150656 (0.45GB), Host Name:server.com
Witness:
########-####-####-####-############
Component State: ACTIVE, Address Space(B): 0 (0.00GB), Disk UUID:
########-####-####-####-############
, Disk Name: naa.###############
:2 Votes: 1, Capacity Used(B): 12582912 (0.01GB), Physical Capacity Used(B): 4194304 (0.00GB), Host Name:server.com
Type: vdisk
Path: /vmfs/volumes/vsan:
###############
-########-##############
/########-####-####-####-############
/VCSA####_6.vmdk (Exists) Group UUID:
########-####-####-####-############
Directory Name: N/A
9. In vSAN 6.6 and above, it is critical to maintain vSAN virtual networking unicast communication between hosts on the vSAN network. Please see KB for configuration details.
Configuring vSAN Unicast networking from the command line
10. Set the following configuration via CLI on each ESXi vSAN Cluster Member Host to be migrated.
esxcfg-advcfg -s 1 /VSAN/IgnoreClusterMemberListUpdates
This will ensure when the hosts are disconnected from the original vCenter, they will not lose their unicast configurations and therefore their connectivity between hosts. If this step is skipped, the unicast address table will be cleared and data availability will be affected temporarily until the table is manually repopulated.
11. Run the command "
esxcli vsan cluster get"
from any hosts in the cluster and take note of the host count to ensure the count matches before and after the migration.
12. Disconnect one host from vCenter by right clicking the host and using the disconnect option. (**Not applicable if old vCenter is unavailable)
13. Remove the host from inventory. Make sure to follow KB 337625 if using a vDS (**Not Applicable if old vCenter is unavailable)
14. Add the hosts only to the DataCenter level of the new vCenter inventory. Do NOT add the hosts directly to the new vSAN enabled cluster otherwise, the host will be forced into Maintenance Mode. (Avoiding Maintenance Mode is key for a seamless downtime migration.)
15. Once the host is registered in the new vCenter inventory, under the DataCenter Inventory Object, it can safely be dragged and dropped into the new vSAN enabled cluster from within the vCenter UI.
16. Before moving on to the next host, verify there is host-to-host vSAN communication. Run the command "esxcli vsan cluster get" from ESXi CLI and take note of the host count to ensure the count matches.
It is good to practice this command each time a host is disconnected, moved, and connected. If the host is not communicating, i.e. the host count is less than expected then it is highly likely a critical step has been missed. Please review the steps and ensure each host's unicast tables are complete and host-to-host communication on the vSAN network is fully functional.
To verify unicast agent table, use below command -
esxcli vsan cluster unicastagent list
17. Once the first host is moved and connectivity checks are performed, move forward with the remaining hosts. Please be certain to follow the steps carefully and check the cluster membership along the way to prevent any data availably impacts.
18. Once all hosts have been added to the new cluster and access to the VMs has been verified, revert the hosts' unicast table configuration. This procedure should be completed at the completion of the migration, and before the hosts are rebooted next. Host rebooting is not necessary for the procedure's success.
esxcfg-advcfg -s 0 /VSAN/IgnoreClusterMemberListUpdates
19. Again verify host and VM connectivity again by pinging the new vCenter from the management interface on each host.
20. Apply the vSAN storage policy to the VMs.
Click OK. Assuming the policy is consistent with the old policy and all the hosts have remained in a healthy state, a resync shouldn't be triggered. It is recommended to perform this action, one at a time. The goal is to move gradually to prevent creating a major resync in the cluster.
21. Repeat this action for the remaining VMs. When performed correctly, there will be no reconfigurations/resyncs. If the VMs start resyncing then there is an inconsistency in the ruleset from the original to the new storage policies. Please allow the resync to complete before proceeding.
22. Verify by browsing to Menu, Policies and Profiles, VM Storage Policies, desired storage policy, and VM Compliance. All the VMs should be shown.