VCF upgrade to 4.5 failed for THIRD_PARTY stage of SDDC component with error "Tag not found for transport zone"

Products

• VMware Cloud Foundation

Issue/Introduction

To resolve the tag not found.

Symptoms:

• VCF upgrade to 4.5 failed for THIRD_PARTY stage of SDDC component with error "Tag not found for transport zone"
• Log located at /var/log/vmware/vcf/lcm/thirdparty/upgrades/54317adc-163d-4a84-b7c9-25a7066fee1c/sddcmanager_migration_app_upgrade_2022-11-22_03-01-08.log shows similar below entries

2022-11-22T03:01:58.819+0000 DEBUG [vcf_migration,0000000000000000,0000[c.v.v.c.f.p.n.h.NsxtCommonOperations,pool-5-thread-1] Filtering Overlay Transport Zone tagged with the name vcf and scope vcf-orchestration

2022-11-22T03:01:58.840+0000 DEBUG [vcf_migration,0000000000000000,0000] [c.v.v.c.n.s.c.c.ApiConnection,pool-5-thread-1] Closed ApiClient connection.

2022-11-22T03:01:58.848+0000 ERROR [vcf_migration,0000000000000000,0000] [c.v.e.s.o.model.error.ErrorFactory,pool-5-thread-1] [VLJ4FN] VCF_ERROR_INTERNAL_SERVER_ERROR Invocation of prefix '' part of task ModifyIsUsedByNsxtForMgmtClusterVdsAction in plugin DriftPlugin failed with exception.

com.vmware.evo.sddc.common.core.error.InternalServerErrorException: Invocation of prefix '' part of task ModifyIsUsedByNsxtForMgmtClusterVdsAction in plugin DriftPlugin failed with exception.

    at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.invokeMethod(ProcessingTaskSubscriber.java:447)

    at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.processTask(ProcessingTaskSubscriber.java:520)

    at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.accept(ProcessingTaskSubscriber.java:124)

    at jdk.internal.reflect.GeneratedMethodAccessor565.invoke(Unknown Source)

    at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

    at java.base/java.lang.reflect.Method.invoke(Method.java:566)

    at com.google.common.eventbus.Subscriber.invokeSubscriberMethod(Subscriber.java:88)

    at com.google.common.eventbus.Subscriber$1.run(Subscriber.java:73)

    at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)

    at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)

    at java.base/java.lang.Thread.run(Thread.java:829)

Caused by: com.vmware.vcf.common.nsxt.services.exceptions.NsxtServiceException: Overlay Transport zone configuration with the tag vcf is not found

    at com.vmware.vcf.common.fsm.plugins.nsxt.helpers.NsxtCommonOperations.getOverlayTransportZone(NsxtCommonOperations.java:847)

    at com.vmware.vcf.common.fsm.plugins.nsxt.helpers.NsxtUtils.getHostSwitchFromTransportNode(NsxtUtils.java:536)

    at com.vmware.vcf.migration.actions.ModifyIsUsedByNsxtForMgmtClusterVdsAction.execute(ModifyIsUsedByNsxtForMgmtClusterVdsAction.java:127)

    at com.vmware.vcf.migration.actions.ModifyIsUsedByNsxtForMgmtClusterVdsAction.execute(ModifyIsUsedByNsxtForMgmtClusterVdsAction.java:33)

    at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionState.invoke(FsmActionState.java:62)

    at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionPlugin.invoke(FsmActionPlugin.java:159)

    at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionPlugin.invoke(FsmActionPlugin.java:144)

    at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.invokeMethod(ProcessingTaskSubscriber.java:400)

    ... 10 common frames omitted

• 2022-11-22T03:01:58.870+0000 ERROR [vcf_migration,0000000000000000,0000] [c.v.v.i.b.i.TranslationMessage,pool-5-thread-1] Can't find resource for bundle java.util.PropertyResourceBundle, key VCF_ERROR_INTERNAL_SERVER_ERROR.remedy

Note:The preceding log excerpts are only examples.Date,time and environmental variables may vary depending on your environment.

Environment

Vmware Cloud Foundation 4.5

Cause

In VCF 4.5, we have added a step into the configuration drift which detects the VDS used for NSXT overlay traffic for management domain and updates the VCF inventory for the same. This action failed here as the VCF tags were not found in the overlay transport zone. If the bringup is run using VCF 4.3, the overlay transport zone will have the tags added to it. If it's missing then it is due to manual intervention. If an overlay transport zone is created outside of VCF , the tags must be manually added.

Resolution

To resolve this issue add the tag to the overlay transport zone - tagName:vcf, tagScope:vcf-orchestration, by logging into the NSXT manager.