Error: "entity not found: got 0 Edge Gateways by name '<EDGE_NAME>'" when creating a load balancer service

Products

• VMware Cloud Director

Issue/Introduction

• Attempting to create a load balancer service in a Kubernetes cluster deployed by VMware Cloud Director Container Service Extension (CSE) fails.
• Attempting to install Contour in a Kubernetes cluster deployed by VMware Cloud Director Container Service Extension (CSE) fails.
• Error: "CreateLoadbalancerError" can be seen in the Kubernetes Container Clusters UI in the Cluster Information page, in the Events tab:

Event Details
Name:   CreateLoadbalancerError
Type:   Error
Time:   <DATE>
Resource Name:  <RESOURCE_NAME>
Detailed Error: unable to create load balancer. err [unable to determine if gateway [<EDGE_NAME>] is using Ip Spaces or not. error [[ENF] entity not found: got 0 Edge Gateways by name '<EDGE_NAME>']]

• IP Spaces are used with the Edge Gateway and Routed Organization VDC Network to which the Kubernetes cluster is connected.

Environment

• VMware Cloud Director Container Service Extension 4.2.1
• VMware Cloud Director 10.5.1.1
• Kubernetes External Cloud Provider for VMware Cloud Director (CPI) 1.6

Cause

This issue occurs if the Edge Gateway and Routed Organization VDC Network are scoped to a Data Center Group in VMware Cloud Director (VCD), IP Spaces are in use and no load balancer IP address is provided when creating the service.

To confirm if the Edge Gateway and Routed Organization VDC Network are part of a Data Center Group, view the Scope column visible in the Tenant portal of Cloud Director in Networking > Edge Gateways and Networking > Networks.

To view the Data Center Groups in the Tenant portal of Cloud Director navigate to Networking > Data Center Groups.

Resolution

This issue is resolved in the Kubernetes External Cloud Provider for VMware Cloud Director 1.6.1 release.

To configure CSE to use this CPI 1.6.1 version when creating Kubernetes clusters, update the CSE server configuration through the Cloud Director Provider portal as per the documentation, Update Server Configuration.

Workaround

To workaround this issue deploy the Kubernetes clusters to a Routed Organization VDC network and not a Data Center Group network.

Alternatively to workaround this issue specify a load balancer IP when creating a load balancer service instead of allowing one to be automatically assigned.

To determine the IPs available review the specific IP Space used with the Edge Gateway and Routed Organization VDC network to which the Kubernetes cluster is connected in the Tenant portal of Cloud Director in Networking > IP Spaces > IP Space Name > Floating IPs.

Please see the example workarounds below for different operations which request a load balancer service in a Kubernetes cluster deployed by CSE.
NOTE: These examples are provided for reference only and other changes may be required for the Kubernetes cluster in question.

Setting a load balancer IP when creating a Kubernetes load balancer service using YAML:

1. Configure the desired load balancer service in a YAML file and use the loadBalancerIP option to specify the desired IP address:

   apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  ports:
    - protocol: TCP
      port: 80
      targetPort: 80
  externalTrafficPolicy: Cluster
  loadBalancerIP: #.#.#.#
  type: LoadBalancer

2. Apply this YAML file to create the desired load balancer service in the Kubernetes cluster:

   kubectl apply -f my-service.yaml

Setting a load balancer IP when creating a Kubernetes load balancer service to expose an app:

1. When using the kubectl expose command to create a load balancer service use the --load-balancer-ip option to specify the desired IP address:

   kubectl expose deployment my-deployment --port=80 --type=LoadBalancer --load-balancer-ip=#.#.#.#

Setting a load balancer IP when installing Contour in a Kubernetes cluster deployed by CSE:

1. Configure the desired YAML file, for example contour-data-values.yaml, for the Contour package and use the loadBalancerIP option to specify the desired IP address:

   infrastructure_provider: vsphere
namespace: tanzu-system-ingress
contour:
  configFileContents: {}
  useProxyProtocol: false
  replicas: 2
  pspNames: "vmware-system-restricted"
  logLevel: info
envoy:
  service:
    type: LoadBalancer
    annotations: {}
    externalTrafficPolicy: Cluster
    disableWait: false
    loadBalancerIP: #.#.#.#
  hostPorts:
    enable: true
    http: 80
    https: 443
  hostNetwork: false
  terminationGracePeriodSeconds: 300
  logLevel: info
certificates:
  duration: 8760h
  renewBefore: 360h

2. Install Contour using this configuration file as per the documentation, Install Contour in Workload Clusters Deployed by a Standalone Management Cluster.

 

Additional Information

Please also see the KB article for the related issue when creating Kubernetes clusters, Error: "entity not found: got 0 Edge Gateways by name '<EDGE_NAME>'" when creating a Kubernetes cluster.