How to Configure Proxy in VKSM Cluster-Agent-Service

Products

• VCF Automation

Issue/Introduction

The procedure outlined in this document is intended to be executed by a VCFA Provider Admin persona. To that effect, below are the prerequisites:

• Access to the VMSP/VCFA appliance cluster via kubectl with administrative privileges.
• The VCFA internal proxy configuration must be available as secrets within the cluster (typically created during the initial VCFA deployment). This can be confirmed by running the below command 
  $ kubectl -n prelude get secrets curl.proxy.https-proxy curl.proxy.proxy-exclude
  NAME                       TYPE     DATA   AGE
  curl.proxy.https-proxy     Opaque   1      xxd
  curl.proxy.proxy-exclude   Opaque   1      xxd

 

Symptoms:

Without the correct proxy configuration, VKS clusters will not show up as healthy in the Kubernetes management UI page within the VCFA UI.

Logs in the cluster-agent-service pod in the VMSP cluster's prelude namespace will indicate a network timeout. This command can be used to inspect the cluster-agent-service pods’ logs.

kubectl -n prelude logs deploy/cluster-agent-service-server

For example:

JSON

{"component":"server-serve-grpc","error":"projects.packages.broadcom.com/vsphere/vksm/extensions/9.0.1-0-24965341/tmc-bootstrapper/manifest:latest was not fetched: could not load image source: get image from the registry: Get \"https://projects.packages.broadcom.com/v2/\": dial tcp xx.xxx.xxx.x:443: i/o timeout","grpc.method":"Create","grpc.request.deadline":
"YYYY-MM-DDTHH:MM:SSZ","grpc.service":"vksm.private.v1alpha1.cluster.agent.AgentResourceService","grpc.start_time":
"YYYY-MM-DDTHH:MM:SSZ","level":"error","msg":"failed to get spec for tmc-bootstrapper","peer.address":"xxx.xx.x.xx:37770","request-id":"xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx","request.kind":"unary","span.kind":"server","subcomponent":"requests","system":"grpc","time":"YYYY-MM-DDTHH:MM:SSZ","uid":""}

Environment

VMware Cloud Foundation Automation (VCFA) 9.0.1

Cause

This issue occurs in environments where there is no direct network connectivity between the VCFA appliance cluster and the image registry, "project.packages.broadcom.com"

In such air-gapped or restricted environments, the cluster-agent-service must be configured to use the internal VCFA proxy to pull necessary images and manifests.

Resolution

To resolve this issue, you must patch the cluster-agent-service-server deployment in the prelude namespace to include the necessary proxy environment variables. These variables will point to the pre-existing VCFA internal proxy secrets.

Step 1: Accessing the VMSP Cluster

Follow the standard procedure to obtain kubectl access to the VMSP cluster:
SSH into the VCFA endpoint.
Verify access: kubectl get pods -n prelude

Step 2: Apply the Proxy Configuration Patch

Run the following command to patch the deployment with the proxy settings:

Bash

kubectl patch deployment cluster-agent-service-server -n prelude --type=strategic -p='
spec:
  template:
    spec:
      containers:
      - name: cluster-agent-service
        env:
        - name: http_proxy
          valueFrom:
            secretKeyRef:
              key: curl.proxy.https-proxy
              name: curl.proxy.https-proxy
        - name: https_proxy
          valueFrom:
            secretKeyRef:
              key: curl.proxy.https-proxy
              name: curl.proxy.https-proxy
        - name: no_proxy
          valueFrom:
            secretKeyRef:
              key: curl.proxy.proxy-exclude
              name: curl.proxy.proxy-exclude
        - name: HTTP_PROXY
          valueFrom:
            secretKeyRef:
              key: curl.proxy.https-proxy
              name: curl.proxy.https-proxy
        - name: HTTPS_PROXY
          valueFrom:
            secretKeyRef:
              key: curl.proxy.https-proxy
              name: curl.proxy.https-proxy
        - name: NO_PROXY
          valueFrom:
            secretKeyRef:
              key: curl.proxy.proxy-exclude
              name: curl.proxy.proxy-exclude'

Validation:

After applying the patch, verify the fix:

1. Monitor Pod Restart: Ensure the cluster-agent-service-server pod restarts successfully.
2. Check Logs: Verify that logs no longer show i/o timeout errors when connecting to projects.packages.broadcom.com.
3. UI Status: Confirm that VKS clusters now appear as "Healthy" in the Kubernetes management UI page of the VCFA console.

NOTE: Above resolution steps must be applied again post upgrading the VCFA environment to version 9.0.2

Additional Information

Disclaimer: Broadcom Support creates these articles to help you with common configuration questions. If you are unsure about applying these changes, please contact Broadcom Support for assistance.