NAPP deployment fails when proxy server is configured on Kubernetes nodes

Products

• VMware vDefend Firewall with Advanced Threat Prevention

Issue/Introduction

Symptoms:
NAPP deployment is failing when Kubernetes API server cannot be reached when a proxy server is configured on Kubernetes nodes
 
There are issues in creating some Kubernetes objects such as secrets. For example, the below command will not list the ca-key-pair secret:
kubectl get secrets --all-namespaces | grep 'ca-key-pair'

Cause

With proxy configured, Kubernetes nodes timeout when trying to communicate across the internet to API server.

Resolution

To allow NAPP deployment to work in this scenario, remove proxy configurations so that the k8s cluster can access the internet directly. The process for removing the proxy configuration will depend on how it was configured.
 
Option 1) If the proxy was directly configured in the containerd config files, then the containerd config file should be changed from:
 
Environment="NO_PROXY=127.0.0.1,10.x.x.x/8,172.x.x.x/12,192.x.x.x/16,localhost,.imatech.intra"
 
to:
 
Environment="NO_PROXY=127.0.0.1,10.x.x.x/8,172.x.x.x/12,192.x.x.x/16,localhost,.xxxxx.xxxxxx,*.kube-system.svc.cluster.local,*.svc.cluster.local,*.cluster.local,*.svc"
 
Then, restart the containerd service.
 
 
Option 2) If proxy was configured in k8s manifest files (/etc/kubernetes/manifests/kube-apiserver.yaml, /etc/kubernetes/manifests/kube-controller-manager.yaml,  /etc/kubernetes/manifests/kube-scheduler.yaml) as below:
cat /etc/kubernetes/manifests/kube-apiserver.yaml
...
env:
- name: https_proxy
value: http://proxy.xxxxxx.xxx:3128
- name: http_proxy
value: http://proxy.xxxxxx.xxx:3128
- name: ftp_proxy
value: http://proxy.xxxxxx.xxx:3128
 

Then the following entry needs to be added under the "env:" section in each .yaml file:

...
env:
- name: no_proxy
value: "*.kube-system.svc.cluster.local,*.svc.cluster.local,*.cluster.local,*.svc"
- name: https_proxy
value: http://proxy.xxxxxx.xxx:3128
- name: http_proxy
value: http://proxy.xxxxxx.xxx:3128
- name: ftp_proxy
value: http://proxy.xxxxxx.xxx:3128
 
Then, restart the kubelet service.
 
 
Option 3) If the Kubernetes cluster was brought up using any third-party tool, then the configuration switches in that tool should be used to get the same "NO_PROXY" configuration.

Additional Information

Important Ports to be enabled  : https://ports.broadcom.com/home/NSX