Cluster creation failure in sddc manager workload domain with error "Multiple overlay-transportzone with the tag VCF and scope VCF-Orchestration found."

Products

• VMware SDDC Manager

Issue/Introduction

• While creating cluster in workload domain, workflow fails at Validate Overlay TransportZone subtask
• /var/log/vmware/vcf/domainmanager/domainmanager.log reports below error:
  
  YYYY-MM-DDTHH:MM:SS.ERROR [vcf_dm,6#############################3,c081] [c.v.e.s.o.model.error.ErrorFactory,dm-exec-17]  [Q#####] MULTIPLE_OVERLAY_TRANSPORT_ZONES_FOUND Multiple Transport zones (GLH-Overlay-TZ, nsx-overlay-transportzone) with the tag vcf and scope vcf-orchestration found in NSX cluster <NSX_CLUSTER_FQDN>
com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: Multiple Transport zones (<CLUSTER_OVERLAY_TZ>, <NSX_OVERLAY_TZ>) with the tag vcf and scope vcf-orchestration found in NSX cluster <NSX_CLUSTER_FQDN>
        at com.vmware.vcf.common.fsm.plugins.nsxt.action.ValidateOverlayTransportZoneAction.execute(ValidateOverlayTransportZoneAction.java:105)
        at com.vmware.vcf.common.fsm.plugins.nsxt.action.ValidateOverlayTransportZoneAction.execute(ValidateOverlayTransportZoneAction.java:33)
        at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionState.invoke(FsmActionState.java:62)
        at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionPlugin.invoke(FsmActionPlugin.java:159)
        at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionPlugin.invoke(FsmActionPlugin.java:144)
        at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.invokeMethod(ProcessingTaskSubscriber.java:400)
        at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.processTask(ProcessingTaskSubscriber.java:520)
        at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.accept(ProcessingTaskSubscriber.java:124)
        at jdk.internal.reflect.GeneratedMethodAccessor827.invoke(Unknown Source)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:569)
        at com.google.common.eventbus.Subscriber.invokeSubscriberMethod(Subscriber.java:85)
        at com.google.common.eventbus.Subscriber.lambda$dispatchEvent$0(Subscriber.java:71)
        at com.vmware.vcf.common.tracing.TraceRunnable.run(TraceRunnable.java:59)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
        at java.base/java.lang.Thread.run(Thread.java:840)
YYYY-MM-DDTHH:MM:SS DEBUG [vcf_dm,6#################################3,c081] [c.v.e.s.o.c.ProcessingTaskSubscriber,dm-exec-17]  Collected the following errors for task with name ValidateOverlayTransportZoneAction and ID 0###############################e: [ExecutionError [errorCode=null, errorResponse=LocalizableErrorResponse(messageBundle=com.vmware.vcf.common.fsm.plugins.nsxt.messages)]]
YYYY-MM-DDTHH:MM:SS ERROR [vcf_dm,6################################0,c0ce] [c.v.e.s.o.c.ProcessingOrchestratorImpl,dm-exec-17]  Stopping task on first failure
YYYY-MM-DDTHH:MM:SS WARN  [vcf_dm,6################################0,c0ce] [c.v.e.s.o.c.ProcessingOrchestratorImpl,dm-exec-17]  Processing State completed with failure
YYYY-MM-DDTHH:MM:SS INFO  [vcf_dm,6################################4,fa7d] [c.v.e.s.o.core.OrchestratorImpl,dm-exec-17]  An error occurred in Execution ID 0#################################8. Transferring to the onError states.

 

Environment

VCF 5.2

Cause

This error occurs because VMware Cloud Foundation (VCF) orchestration logic expects a unique Transport Zone (TZ) to be tagged for its use. When it finds two (or more) TZs with the exact same vcf / vcf-orchestration tag-scope pair, the automation engine (SDDC Manager) enters a conflict state and task fails.

Resolution

Remove the tag from one overlay transport zone.

Note: Remove the tag only from  CLUSTER_OVERLAY_TZ.

DO NOT remove the tag from NSX_OVERLAY_TZ

Steps to remove tags from CLUSTER_OVERLAY_TZ.

1. Log in to the NSX Manager UI.
2. Navigate to System > Fabric > Transport Zones.
3. Locate the Transport Zone you want to modify (e.g., the duplicate <CLUSTER_OVERLAY_TZ>).
4. Click on the Number displayed in the Tags column for that specific row.
5. A "Manage Tags" window will pop up.
6. Find the row where the Scope is vcf-orchestration and the Tag is vcf.
7. Click the X (delete icon) on the far right of that specific tag row.
8. Click Save.

Return to SDDC manager and restart the cluster creation task.