Contour pods unable to pull image from projects.packages.broadcom.com

Products

• VMware NSX
• VCF TGW

Issue/Introduction

Contour pods fail to pull images from projects.packages.broadcom.com due to a network connection failure. The supervisor control plane VM is unable to connect to the external repository via the eth1 interface.

You will see an error similar to the following for the failing pods:

error="failed to do request: Head \"https://projects.packages.broadcom.com/v2/vsphere/supervisor/contour/1.33.1/contour/manifests/sha256:<sha256sum>\": dial tcp <IP Address>:443: connect: network is unreachable"

Additionally, running a curl test from the supervisor control plane VM confirms the unreachable network state:

root@<hostname> [ ~ ]# curl -v -k --interface eth1 https://projects.packages.broadcom.com
* Host projects.packages.broadcom.com:443 was resolved.
* IPv6: (none)
* IPv4: <IP Address>, <IP Address>, <IP Address>
* Trying <IP Address>:443...
* socket successfully bound to interface 'eth1'
* connect to <IP Address> port 443 from <IP Address> port 40046 failed: Network is unreachable
...
* Failed to connect to projects.packages.broadcom.com port 443 after 4 ms: Could not connect to server
* closing connection #0
curl: (7) Failed to connect to projects.packages.broadcom.com port 443 after 4 ms: Could not connect to server

Environment

• VMware vSphere (Supervisor)
• Contour 1.33.1
• VMware VCF Networking

Cause

The NSX environment is missing a required route. Because the Virtual Routing and Forwarding (VRF) instance does not have a defined path for this outbound traffic, it drops the packets directed toward the public repository. An NSX traceflow confirms the packet is "Dropped for no route found" on the inter-vrf-port.

Resolution

To resolve this issue, you must configure a static or dynamic route in NSX to allow north/south traffic to traverse properly:

1. Identify the required network path for your outbound external repository traffic.

2. Create a static or dynamic route within NSX to provide the VRF with a defined path to external addresses.

   • Staic Routing: Create an NSX Inter-VRF Static Route with Next Hop as the Destination VRF

   • OSPF Routing: Configure an NSX OSPF

   • BGP Routing: Configure BGP

3. Re-run an NSX packet traceflow or a command-line curl test to verify that the traffic successfully traverses the inter-vrf-port.