When the VCF configuration drift bundle is applied, the upgrade fails at the task "Collect inventory data in a form useful for edge cluster processing"

Products

• VMware SDDC Manager

Issue/Introduction

An upgrade is being done from a previous version of VCF (example: 4.4.1) to 4.5.2

The SDDC Manager has been upgraded to 4.5.2

The next step is to apply the configuration drift bundle for VMware Cloud Foundation 4.5.2.0

This activity fails within one to two minutes of being started, with the following UI error being shown (xxxxx corresponds to the management domain name) - 

xxxxx domain failed upgrade at Mmm dd, yyyy, hh:mm:ss. Please resolve the above upgrade failure for this bundle before applying any other available buncle

 

The following is observed in the SDDC Manager migration app upgrade log -

/var/log/vmware/vcf/lcm/thirdparty/upgrades/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx/sddcmanager-migration-app/logs/sddcmanager_migration_app_upgrade_yyyy-mm-dd_hh-mm-ss.log

DEBUG [vcf_migration,0000000000000000,0000] [c.v.e.s.o.c.ProcessingTaskSubscriber,pool-5-thread-16]  Invoking task PrepareNsxtEdgeClusterDriftInputAction. Description: Collect inventory data in a form useful for edge cluster processing, Plugin: DriftPlugin, ParamBuilder null, Input map: {credentials=ConfigDrift____0__credentials, inventory=ConfigDrift____0__inventory, domainId=domainId}, Id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx ...
INFO  [vcf_migration,0000000000000000,0000] [c.v.v.m.a.PrepareNsxtEdgeClusterDriftInputAction,pool-5-thread-16]  Starting preparation for NSX-T Edge Cluster Drift configuration
ERROR [vcf_migration,0000000000000000,0000] [c.v.e.s.o.model.error.ErrorFactory,pool-5-thread-16]  [XXXXXX] VCF_ERROR_INTERNAL_SERVER_ERROR Invocation of prefix '' part of task PrepareNsxtEdgeClusterDriftInputAction in plugin DriftPlugin failed with exception.
com.vmware.evo.sddc.common.core.error.InternalServerErrorException: Invocation of prefix '' part of task PrepareNsxtEdgeClusterDriftInputAction in plugin DriftPlugin failed with exception.
.
.
Caused by: java.lang.NullPointerException: null
.
.
ERROR [vcf_migration,0000000000000000,0000] [c.v.v.i.b.i.TranslationMessage,pool-5-thread-16]  Can't find resource for bundle java.util.PropertyResourceBundle, key VCF_ERROR_INTERNAL_SERVER_ERROR.remedy
DEBUG [vcf_migration,0000000000000000,0000] [c.v.e.s.o.c.ProcessingTaskSubscriber,pool-5-thread-16]  Collected the following errors for task with name PrepareNsxtEdgeClusterDriftInputAction and ID xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx: [ExecutionError [errorCode=null, errorResponse=LocalizableErrorResponse(messageBundle=com.vmware.evo.sddc.common.core.error.messages)]]
DEBUG [vcf_migration,0000000000000000,0000] [c.v.e.s.o.c.ProcessingTaskSubscriber,pool-5-thread-16]  Invoking task PrepareNsxtEdgeClusterDriftInputAction.UNDO Description: Collect inventory data in a form useful for edge cluster processing, Plugin: DriftPlugin, ParamBuilder null, Input map: {credentials=ConfigDrift____0__credentials, inventory=ConfigDrift____0__inventory, domainId=domainId}, Id: xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx ...

 

Environment

VCF 4.5.2

Cause

This issue is caused by a cluster being in a DEACTIVATING status in the cluster table in the platform database on SDDC Manager.

This can be a result of the cluster removal workflow having failed or only partially completing, or the cluster having been manually deleted from the vSphere Client and NSX.

In effect, the cluster in a DEACTIVATING state is a stale cluster - and this is blocking the workflow.

Resolution

Please open a support case with VMware by Broadcom technical support in order to have a resolution implemented.