Networking Requirements for downloading VKS Supervisor Services

Products

• VMware vSphere Kubernetes Service

Issue/Introduction

Table of Contents

• Overview
• Verifying Connectivity
  • NSX
  • VDS

Overview

• When a Supervisor Service is installed, the kapp-controller running on the Supervisor control plane VMs pulls the Carvel imgpkg bundle that defines the service from projects.packages.broadcom.com over HTTPS. This bundle is pulled on the Supervisor management network.
  
  
• kapp-controller then deploys the YAML extracted from the service bundle.
  
  
• This results in a PodVM being created that will pull its container image(s) via the image-fetcher component running on each ESX host on the Supervisor workload network. If you are using a vSphere Distributed Switch (VDS) networking stack solution (NSX Advanced Load Balancer or HAProxy) the primary workload will be used to pull the container images.
  
  
• For services that run on the Supervisor control plane (vSphere Kubernetes Service, Velero, etc.), the container images will be pulled from the Supervisor control plane VMs over the management network.

 

Verifying Connectivity

NSX

You can use the NSX traffic analysis tool Traceflow to verify connectivity from your WCP environment to projects.packages.broadcom.com. Connect to your NSX Manager using a web browser and select:

1. Plan & Troubleshoot
   
   
2. Traffic Analysis under Troubleshooting Tools
   
   
3. Get Started in the Traceflow section

Set the following and then press Trace:

1. Protocol Type: TCP
   
   
2. Destination Port: 443
   
   
3. Source Type: Port/Interface
   
   
4. Port: A port connected to one of your Supervisor control plane VMs. Using the information from the Overview, determine if you need to choose a port from the workload or management network.
   
   
   1. You can verify the IPs by using the vSphere Client, find a SupervisorControlPlaneVM(x) and viewing its IP addresses:
      
      
      
      
   2. You can view your Supervisor's network settings by going to:
      
      
      1. vSphere Client Menu
         
         
      2. Workload Management (vSphere 8) / Supervisor Services (vSphere 9)
         
         
      3. Supervisor tab
         
         
      4. Select the relevant supervisor
         
         
      5. Configure tab
         
         
      6. Network under the Supervisor section
         
         
      7. For the management network:
         
         
         1. Expand Management Network
            
            
         2. View the Starting IP Address
            
            
      8. For the workload network:
         
         
         1. Expand Workload Networks
            
            
         2. In the datagrid, view the IP address range for the network that has the Primary label
            
            
5. Destination Type: IP - Mac
   
   
6. Destination Layer: Layer 3
   
   
7. Destination IP: IP address of projects.packages.broadcom.com

Once the trace is complete, scroll to the bottom and check the Observation Type column. It should have a green Delivered label if the request was successful. If it wasn't successful, you will need to address the network configuration in your environment.

VDS

1. ssh to your vCenter server
   
   
2. Run /usr/lib/vmware-wcp/decryptK8Pwd.py
   

   Read key from file
   
   Connected to PSQL

   
   Cluster: domain-<ID>

   IP: 192.168.1.201

   PWD: <PASSWORD>

   
   
3. ssh to the IP listed above
   
   
4. If prompted, enter yes to continue
   
   
5. Enter the PWD value above
   
   
6. Once you've logged into the supervisor control plane VM, test connectivity to projects.packages.broadcom.com by running one of the following commands. A response of 0 (listed directly below the curl command) indicates a success. Any other number indicates a failure:
   
   
   1. Test from the management network:
      

      curl -k --interface eth0 https://projects.packages.broadcom.com:443 ; echo $?
      0

       

   2. Test from the workload network:
      

      curl -k --interface eth1 https://projects.packages.broadcom.com:443 ; echo $?
      0

Environment

vSphere Kubernetes Service

Resolution

Configure your network so that the WCP management and workload networks can reach the following addresses:

• wp-content.vmware.com
• *.tmc.cloud.vmware.com
• projects.registry.vmware.com
• projects.packages.broadcom.com
• jfrog-prod-usw2-shared-oregon-main.s3.amazonaws.com

Additional Information

• Supervisor Services
• kapp-controller
• imgpkg