Error: "Tier1 '/infra/tier-1s/########-####-####-####-############' cannot be deleted as Virtualservice 'VCD-LoadBalancer-########-####-####-####-############' associated with it" when attempting to enable Load Balancer Transparent Mode on an Edge Gateway

Products

• VMware Cloud Director

Issue/Introduction

• Cannot activate Transparent Mode under Edge Gateways > Edge Name > Load Balancer > General Settings in the Cloud Director Provider or Tenant Portals.
• Task fails in the Cloud Director UI with an error of the form:
  
  Tier1 '/infra/tier-1s/########-####-####-####-############' cannot be deleted as Virtualservice 'VCD-LoadBalancer-########-####-####-####-############-<virtual_service_name>' associated with it" when attempting to enable Load Balancer Transparent Mode on an Edge Gateway.
  
  
• The /opt/vmware/vcloud-director/logs/vcloud-container-debug.log on the Cloud Director Cells show an error of the form:
  
  <timestamp> | ERROR    | task-service-activity-pool-#### | FutureUtil                     | Task future is not complete. | requestId=########-####-####-####-############,request=PUT https://<vcd_address>/cloudapi/1.0.0/edgeGateways/urn:vcloud:gateway:########-####-####-####-############/loadBalancer,requestTime=<timestamp>,remoteAddress=#.#.#.#:#,userAgent=...,accept=application/json;version 40.0.0-alpha vcd=########-####-####-####-############,task=########-####-####-####-############ activity=(com.vmware.vcloud.backendbase.management.system.TaskActivity,urn:uuid:########-####-####-####-############)
com.vmware.vcloud.common.network.NsxAlbApiException: Tier1 '/infra/tier-1s/########-####-####-####-############' cannot be deleted as Virtualservice 'VCD-LoadBalancer-########-####-####-####-############' associated with it.
        at com.vmware.vcloud.fabric.net.nsxalb.NsxAlbRestTemplateErrorHandler.handleError(NsxAlbRestTemplateErrorHandler.java:46)
        at org.springframework.web.client.ResponseErrorHandler.handleError(ResponseErrorHandler.java:63)
        at org.springframework.web.client.RestTemplate.handleResponse(RestTemplate.java:825)
        at org.springframework.web.client.RestTemplate.doExecute(RestTemplate.java:783)
        at com.vmware.vcloud.fabric.net.utils.impl.LoggingRestTemplate.doExecute(LoggingRestTemplate.java:72)
        at org.springframework.web.client.RestTemplate.execute(RestTemplate.java:717)
        at org.springframework.web.client.RestTemplate.put(RestTemplate.java:503)
        at com.vmware.vcloud.networking.model.nsxalbresource.NsxAlbCloudResource.updateCloud(NsxAlbCloudResource.java:67)

Environment

• VMware Cloud Director 10.6.x
• VMware Cloud Director 10.5.x
• VMware Avi Load Balancer

Cause

This issue occurs when the Avi Cloud which is selected for the Edge Gateways NSX-T Cloud has segment_id and tier1_lr_id entries for the Edge Gateway which include and ID only and do not include a full path including /infra/segments/ and /infra/tier-1s/ respectively.
This can be seen when querying the Avi Load Balancer API for details of the Avi Cloud as per the VMware Avi Load Balancer API, Cloud documentation.

GET https://<avi_controller_address>/api/cloud/<cloud_uuid>

For example the following entries are incorrect as they do not include the full path:

{
...
  "name": "vcd-cloud",
  "nsxt_configuration": {
    "automate_dfw_objects": true,
    "automate_dfw_rules": false,
    "data_network_config": {
      "tier1_segment_config": {
        "manual": {
          "tier1_lrs": [
            {
              "segment_id": "12ab####-####-####-####-############",
              "tier1_lr_id": "34cd####-####-####-####-############"
            },
...
}

The current versions of Cloud Director would set and expect the following:

{
...
  "name": "vcd-cloud",
  "nsxt_configuration": {
    "automate_dfw_objects": true,
    "automate_dfw_rules": false,
    "data_network_config": {
      "tier1_segment_config": {
        "manual": {
          "tier1_lrs": [
            {
              "segment_id": "/infra/segments/12ab####-####-####-####-############",
              "tier1_lr_id": "/infra/tier-1s/34cd####-####-####-####-############"
            },
...
}

Resolution

This is a known issue affecting VMware Cloud Director 10.5.x and 10.6.x.

Workaround:

To resolve this issue disable and re-enable the Load Balancer feature on the affected Edge, Enable or Edit the Load Balancer on an NSX
Edge Gateway in the VMware Cloud Director Service Provider Admin Portal.
WARNING: The existing Virtual Services, Pools, Health Monitors and Service Engine Groups on the Edge will need to removed and recreated which will incur downtime.