VCF brownfield import fails during VLAN-backed transport node collection configuration with "ConcurrentChange (statusCode:409)" error

Products

• VMware NSX

Issue/Introduction

When performing a VMware Cloud Foundation (VCF) brownfield import, the import task fails during the configuration of VLAN-backed transport node collections for a cluster.

You will observe the following symptoms across the environment:

• VCF Operations tasks UI : Task failed with a ConcurrentChange transaction conflict error:

  Message: Error in configuring VLAN backed transport node collections
Cause: com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: Error in configuring VLAN backed transport node collections for clusters <CLUSTER_NAME> Error in configuring VLAN backed transport node collections for clusters <CLUSTER_NAME> ConcurrentChange (com.vmware.vapi.std.errors.concurrent_change) (statusCode:409) => { messages = [], data = => {error_message=The object was modified by somebody else. Please retry., httpStatus=CONFLICT, details=Operation failed because of conflicting transaction. Transaction ID: <TRANSACTION_ID> error_code=603, module_name=common-services, error_data= => {STREAM_ID=<STREAM_ID>, CONFLICT_VALUE=java.lang.Error: Unable to find the corresponding key, CONFLICT_KEY_HASH=<CONFLICT_KEY_HASH>, CONFLICT_KEY=uuid { <UUID> } , MAP_NAME=<MAP_NAME>, TRANSACTION_ID=<TRANSACTION_ID>, OFFENDING_ADDRESS=<OFFENDING_ADDRESS>}}, errorType = CONCURRENT_CHANGE } 

• NSX Manager UI: Host preparation progress appears stuck at 100% "Configuring" with an "Unknown" node status. No NSX VIBs are successfully deployed to the ESXi layer.

• vCenter Server UI: The "Import updates" task is permanently stuck at 80%.

Environment

VMware Cloud Foundation 9.0

Cause

This issue is caused by a deadlock in the vCenter Server. The vLCM/VUM (vSphere Lifecycle Manager / vSphere Update Manager) "Import updates" task hangs at 80% due to an internal loopback timeout.

This stuck task blocks metadata synchronization between vCenter Server and NSX Manager. When VCF attempts to configure the transport node collections, the synchronization blocker results in an HTTP 409 CONCURRENT_CHANGE conflict error, as the state of the objects cannot be aligned.

This underlying vCenter Server behavior is documented in Broadcom KB article: KB 421048.

Resolution

To resolve this issue, restart the VMware Update Manager service on the vCenter Server Appliance (VCSA) to clear the deadlocked task and resume synchronization.

1. Log in to the affected vCenter Server Appliance (VCSA) via SSH using root credentials.

2. Enable the BASH shell if prompted by typing shell.

3. Run the following command to restart the vmware-upgrademgr service:
   service-control --restart vmware-upgrademgr

4. Log back into the vSphere Client and verify that the stuck "Import updates" task has cleared.

5. Check NSX Manager to ensure host preparation completes successfully and node status changes to "Up/Success".

6. Retry the VCF brownfield import task from the SDDC Manager UI.

Additional Information

Broadcom KB 421048