vCenter major upgrade (7.x to 8.x) fails on stage 2 of the upgrade process

Products

• VMware vCenter Server

Issue/Introduction

During the upgrade process in doing a major upgrade of vCenter from 7.x to 8.x the upgrade fails at stage 2 - "Data transfer and vCenter Server setup is in progress".

The percentage complete is 50%.

The task is "1. Copy data from source vCenter Server to target vCenter Server".

The error is -

A problem occurred while - Applying network configuration

 

Environment

VCF 7.x

Cause

The subnet default gateway is replying to arp for the IP of the source vCenter VM which was shut down.

An arp for the source vCenter VM IP is being sent out periodically by the "Applying network configuration" workflow on the target vCenter VM.

This can be confirmed using this command -

tcpdump -i any arp

Normally, an arp request to the source vCenter VM should not return any MAC address - given that the source vCenter VM that had that IP is shut down.

In the case where this issue is encountered, the MAC address of the subnet default gateway is being returned in the arp reply.

This should not normally be happening - it is what is preventing the "Applying network configuration" task from completing.

This can happen when Proxy ARP is enabled on the VLAN.

The workflow believes the source vCenter VM is still running.

Resolution

Confirm that Proxy ARP is configured on the VLAN (the customer will need to confirm this for their network infrastructure).

Disable Proxy ARP on the VLAN (the customer will need to do this on their network infrastructure.

This will prevent a MAC address being returned for the source vCenter VM IP when the source vCenter VM is not powered on.

Run this command to confirm that a MAC address is no longer being returned for the source vCenter VM IP once the source vCenter VM is powered off -

tcpdump -i any arp

Once Proxy ARP is disabled on the VLAN, the upgrade is expected to complete successfully.