vCenter Server upgrade failed with error "VCENTER UPGRADE SET REPO" from SDDC Manager UI.

Products

• VMware SDDC Manager
• VMware vCenter Server

Issue/Introduction

• The upgrade workflow fails on SDDC manager UI in the VCENTER UPGRADE SET REPO stage
  
  
  
  
  
• Update failed error message in the update history tab in SDDC manager UI:

• On lcm-debug.log in SDDC manager logs we observing the following errors in the location: /var/log/vmware/vcf/lcm/lcm-debug.log 

Failed to set the repository https://<vcenter_fqdn>/vmware/vcf/bundle/<bundle_id>/update_repo during upgrade for vCenter <vc_fqdn> due to exception 
NotFound (com.vmware.vapi.std.errors.not_found) => { messages = [LocalizableMessage (com.vmware.vapi.std.localizable_message) => { id = com.vmware.appliance.update_error, defaultMessage = %s, args = [Network failure. Check the network settings and try again.], params = <null>, localized = <null> }], data = <null>, errorType = NOT_FOUND }

• Observed the following errors in applmgmt.log in the vCenter Server: /var/log/vmware/applmgmt/applmgmt.log

DEBUG:vmware.appliance.update.update_functions:runCommandAndCheckResult failed: '--xxxx-xx-xx xx:xx:xx--  https://<vcenter_fqdn>/vmware/vcf/bundle/<bundle_id>/update_repo/manifest/manifest-latest.xml

Resolving <proxy_fqdn>... <ip_address>Connecting to <proxy_fqdn>|<ip_address>|:<port>... connected.

Unable to establish SSL connection.

'xxxx-xx-xxTxx:xx:xx AM UTC [xxxxxxx]ERROR:vmware.appliance.update.update_b2b:Got exception while trying discover at URL https://<vcenter_fqdn>/vmware/vcf/bundle/<bundle_id>/update_repo: 

NotFound(messages=[{'id': 'com.vmware.appliance.update_error', 'default_message': '%s', 'args': 

['Network failure. Check the network settings and try again.']}], data=None, error_type='NOT_FOUND')

Environment

VMware SDDC Manager
VMware vCenter Server

Cause

Traffic is blocked from a proxy server preventing the SDDC to set the repository settings to the vCenter.

Resolution

Engage your internal network team to investigate the configuration

Workaround:

• Temporarily disable the vCenter Proxy configuration from the VAMI
• Reboot the VC
• Temporarily disable the SDDC Proxy configuration from the SDDC UI
• Retry the upgrade

Note: In some cases disabling the SDDC Manager Proxy can cause issues with carrying out the upgrade so we had to enable it again.