vCenter Server upgrade from 7.0 to 8.0 fails at 39% with error Upgrade phase timed out after getting stuck at Exporting the VMware Analytics Service data

Products

• VMware vCenter Server

Issue/Introduction

Symptoms:

• During Stage-2 of the upgrade process: Exporting VMware Analytics Service data – stuck at 39%
  • 

 

• Upgrade phase timed out. The time planned for the upgrade phase was 60 minutes. The upgrade phase has already been running for 60 minutes. To extend the default timeout, set environment variable UPGRADE_EXPORT_TIMEOUT
  • 

Environment

VMware vCenter Server 7.0.3
VMware vCenter Server 8.0.x

Cause

This occurs because a push telemetry process has send too many requests with different IDs and that creates too many files in "/var/log/vmware/analytics/prod" (There have been cases with a total of 185k to 375k files).

Resolution

1. Connect to the source vCenter via SSH
2. Run the following command to check the number of files ls /var/log/vmware/analytics/prod | wc -l
3. Download the cleaning_vddk_script.sh script from the attachments or from the https://knowledge.broadcom.com/external/article?legacyId=85116
4. Upload the script to vCenter Server via SCP. You could upload the script to any location (ex: /tmp)
5. Go to the location where you saved the file
6. Make sure the file has "execute" permissions with this command: chmod +x cleaning_vddk_script.sh
7. Run the script: ./cleaning_vddk_script.sh
8. Re-run the stage-2 upgrade again.

Note: By default, the script will apply changes to files older than 180 days. You can pass a parameter to the script in order to use different value such as 90 days, ex:

./cleaning_vddk_script.sh 90