vCenter upgrade fails with Internal error occurs during Export of Vmware vSphere Auto Deploy

Products

• VMware vCenter Server

Issue/Introduction

• vCenter upgrade is failing during the Stage 2 of the upgrade with error:
  
  Internal error occurs during Export of Vmware vSphere Auto Deploy
  
  
  
• vCenter upgrade export logs shows the following error:
  
  yyyy-mm-ddThh:mm:ss.zzzz ERROR autodeploy.upgrade_util Unable to fetch the logs
Traceback (most recent call last):
  File "/tmp/vmware-upgrade-temp-dirE7OeNX1xLQ/tmpatTLqH76j6/payload/component-scripts/autodeploy/upgrade_util.py", line 534, in fetchLogs
    logZip.write(os.path.join(root_dir, name))
  File "/usr/lib/python3.7/zipfile.py", line 1746, in write
    with open(filename, "rb") as src, self.open(zinfo, 'w') as dest:
  File "/usr/lib/python3.7/zipfile.py", line 1473, in open
    return self._open_to_write(zinfo, force_zip64=force_zip64)
  File "/usr/lib/python3.7/zipfile.py", line 1586, in _open_to_write
    self.fp.write(zinfo.FileHeader(zip64))
  File "/usr/lib/python3.7/zipfile.py", line 447, in FileHeader
    len(filename), len(extra))
struct.error: ushort format requires 0 <= number <= (0x7fff * 2 + 1)
yyyy-mm-ddThh:mm:ss.zzzz ERROR autodeploy.autodeploy_upgrader Unable to perform a successful export
Traceback (most recent call last):
  File "/tmp/vmware-upgrade-temp-dirE7OeNX1xLQ/tmpatTLqH76j6/payload/component-scripts/autodeploy/autodeploy_upgrader.py", line 210, in exportData
    fetchLogs(metadata, si, sourceVcVersion, exportDir)
  File "/tmp/vmware-upgrade-temp-dirE7OeNX1xLQ/tmpatTLqH76j6/payload/component-scripts/autodeploy/upgrade_util.py", line 534, in fetchLogs
    logZip.write(os.path.join(root_dir, name))
  File "/usr/lib/python3.7/zipfile.py", line 1746, in write
    with open(filename, "rb") as src, self.open(zinfo, 'w') as dest:
  File "/usr/lib/python3.7/zipfile.py", line 1473, in open
    return self._open_to_write(zinfo, force_zip64=force_zip64)
  File "/usr/lib/python3.7/zipfile.py", line 1586, in _open_to_write
    self.fp.write(zinfo.FileHeader(zip64))
  File "/usr/lib/python3.7/zipfile.py", line 447, in FileHeader
    len(filename), len(extra))
struct.error: ushort format requires 0 <= number <= (0x7fff * 2 + 1)
yyyy-mm-ddThh:mm:ss.zzzz ERROR __main__ Upgrade Phase 'autodeploy:Export' failed. Exception: ushort format requires 0 <= number <= (0x7fff * 2 + 1)

Environment

VMware vCenter Server

Cause

The error occurs while the upgrade workflow attempts to make a copy of the source logs to the destination vCenter.

This is a known issue affecting Python 3.7 where a bad error message occurs when attempting zipping the files.

The problems appears to be related with the log files timestamps.

Resolution

To workaround the issue update all the files timestamps inside the Auto Deploy logs location.

To achieve the task follow the steps below:

1. Log in to the Source vCenter on SSH with root.
2. Type shell to enter bash
3. Navigate to Auto Deploy logs location:
   
   cd /var/log/vmware/rbd/
   
   
4. Update the timestamp using the following command:
   
   touch *
   
   
5. Check all the files have their timestamps updated:
   
   ls -l
   
   
6. Reattempt the upgrade.

Additional Information

To confirm the issue or test if it's fixed you can also run manually the command and observe the output:

python /usr/lib/python3.7/zipfile.py -c /tmp/rbd.zip /var/log/vmware/rbd