VCFA Upgrade subtask "Stage VCF Automation package in VCF services runtime" fails

Products

• VCF Automation
• VCF Operations

Issue/Introduction

VCF Ops upgrade of VCFA shows these errors:

• Internal error occurred. Failed to stage the VCF Component. Please retry the operation or contact support if the issue persists. (VCFMS-STAGE-COMPONENT-002)
• Step vmsp_stage_package failed. Reference Code: _____. Check logs for details.

The "Stage Components" script on VCFA control plane nodes show these errors:

• Command:
  • kubectl -n vmsp-platform logs stage-component-<tag>-execute-script-####
• Errors:
  • ERROR : <tag> Command output: <timestamp> bundle.vmsp.vmware.com/v1alpha1, Kind=Bundle/prelude/vcfa-bundle-9.#.#.#.#####: updated
  • retrying after error: deployment timed out: context deadline exceeded

Environment

• VCF Operations 9.x
• VCF Automation 9.x

Cause

Secure Gateways may be terminating the transfer after 10GB with their default config,

Resolution

Ensure that downloads larger than 10GB are allowed through the network.

Running this download on the VCFA Services Runtime node may be useful to confirm connectivity.
This 9.1 bundle is an example. The particular vcfa-bundle can be found through the upgrade logs on VCFA (in namespace vmsp-platform):

curl -k https://<Fleet-Management-FQDN>/depot-service/content-gateway/PROD/COMP/VRA/vcfa-bundle-9.1.0.0.25370929.tar --output /tmp/vcfa-bundle-9.1.0.0.25370929.tar

Additional Information

See "Maximum amount of data per connection or request" in this document for an example of default configuration that limits downloads to 10GB:

Skyhigh Security: Advanced Settings (for Proxies)