HCX MON: "Change VM Router Location" Stuck at 0%

Products

• VMware HCX

Issue/Introduction

• As part of MON configuration, where a MON is enabled per VM level by setting the Target Router Location, the task gets stuck at 0%
  
  Task: Change VM Router Location
  vm-7###
  Progress: 0%
  Status: Router Location Change Running
  
  
• Login to the HCX Manager as user admin via ssh/cli
  • The app.log at /common/logs/admin shows job getting queued
    
    2026-01-26 16:30:42.906 UTC [InterconnectService_SvcThread-108308, Ent: HybridityAdmin, , TxId: #####-####-#####-########] WARN  c.v.h.i.i.InterconnectIntentUtil- No dependency check needed for this Intent : MODIFY_PR_VMS
    2026-01-26 16:30:42.906 UTC [InterconnectService_SvcThread-108308, Ent: HybridityAdmin, , TxId: #####-####-#####-########] INFO  c.v.v.h.s.i.IntentObserver- Start activity for an intentName :MODIFY_PR_VMS intentId: #####-####-#####-########
    2026-01-26 16:30:42.915 UTC [InterconnectService_SvcThread-108308, Ent: HybridityAdmin, , TxId: #####-####-#####-########] INFO  c.v.v.h.s.i.IntentObserver- Queued job 7f42#####-2b6a-####-b##a-######### with workflow : {"intent":"MODIFY_PR_VMS","jobType":"NetworkStretchJobs","startState":"BEGIN"
    ,"workflowType":"BindRouterEndpointToVms"} and InterconnectTaskId :
    2026-01-26 16:30:43.188 UTC [NetworkStretchService_SvcThread-4555, j: 7f42915b, s: #######, PR, TxId:#####-####-#####-########] INFO  c.v.v.h.networkstretch.AbstractJob- cloud*********{
       "url": "https://10.##.##.##",
    
    
• Subsequent tasks like extension or un-extension of network get stuck.
  
  Task: Deactivating Network Extension
  Progress - 5 %
  Status: Operation timedout in state PROCESS_UPDATE_STRETCHRECORDS_REMOTE
  
  The app.log at /common/logs/admin shows dependency error
  
  2026-01-27 02:38:02.334 UTC [InterconnectService_SvcThread-151, Ent: HybridityAdmin, , TxId: #####-####-#####-########] INFO c.v.v.h.s.i.IntentObserver- Delaying intents [{"_id":"###############","intent":"UNEXTEND_NETWORK","rowType":"INTENT_ROW","intentId":"#####-####-#####-########","isDeleted":false,"enterprise":"DEFAULT","intentData":{"request":{"edgeId":"Tier1","vcTask":"task-245173","vlanId":"X","networkId":"dvportgroup-3###","stretchId":"#####-####-#####-########","endpointId":"#####-####-#####-########","resourceId":"#####-####-#####-########","skipIntent":true,"applianceId":"#####-####-#####-########","connectToLocalRouter":true,"skipStateOnException":true},"intentId":"#####-####-#####-########","secondaryEntityId":"dvportgroup-3###"},"lastUpdated":"2026-01-27T01:39:05.057Z","userContext":{"roles":[],"username":"VSPHERE.LOCAL\\Administrator","enterprise":"DEFAULT","organization":["DEFAULT"],"currentOrganization":"DEFAULT"},"controlState":"NEW","creationDate":"2026-01-27T01:39:05.057Z","creationUser":"VSPHERE.LOCAL\\Administrator","organization":"DEFAULT","lastUpdateUser":"VSPHERE.LOCAL\\Administrator","creationEnterprise":"DEFAULT","creationOrganization":"DEFAULT","interconnectEntityId":["#####-####-#####-########"],"lastUpdateEnterprise":"DEFAULT","interconnectEntityType":"UNEXTEND_NETWORK","lastUpdateOrganization":"DEFAULT"}] by 5 as all new intents are delayed due to being dependent.

Environment

VMware HCX

Cause

The jobs typically get stuck when underlying network issues is present at either the on-premises or cloud sites that prevent the HCX managers from successfully processing the tasks.

Resolution

To resolve this issue, the stuck jobs must be manually marked as 'Failed' or 'Completed' within the HCX database.

Workaround: To ensure that the jobs update its status forcefully, the HCX managers on both on-prem and cloud can be rebooted

However, please note that few jobs may remain stuck even after the reboot and will require manual database cleanup tasks to be executed.

If you encounter this behavior, please collect the HCX log bundles (including database dumps) from both the On-Premise and Cloud managers and open a support request with Broadcom - Creating and managing Broadcom cases