HCX tasks hangs after configuring migration with clearSchedule set to true through API

Products

• VMware HCX

Issue/Introduction

• HCX Site Pairing status stays in Pending after reauthentication
  
  
• Editing or Resyncing a service mesh stuck on 21%
  
  
• Editing Compute Profile or Network Profile fails with:
  "Cannot create/modify Compute Profile <Name> since another operation is already in progress"
  
  

 

• New migrations are queued and don't generate logs in HCX Manager /common/logs/admin/app.log:

  <timestamp> [MobilityMigrationService_SvcThread-178, Ent: HybridityAdmin, , TxId: ##########-####-####-####-##########] INFO  c.v.h.s.m.jobs.MobilityMigrationJob- Migration Requested -> Id: ##########-####-####-####-########## ,  Type: RAV
  <timestamp> [MobilityMigrationService_SvcThread-179, Ent: HybridityAdmin, , TxId: ##########-####-####-####-##########] INFO  c.v.h.s.m.jobs.MigrationTracker- Migration Starting -> Id: ##########-####-####-####-########## ,  Type: RAV
  (No logs after this point)...

   

• Previous migrations indicate that the process never reaches the switchover status, resulting in an excessive amount of logged information. The following log can be found in /common/logs/admin/app.log:

<timestamp> UTC [MobilityMigrationService_SvcThread-3503, Ent: HybridityAdmin, , TxId: ##########-####-####-####-##########] INFO  c.v.h.s.m.jobs.MigrationTracker- Migration Waiting for Maintenance Window -> Id: ##########-####-####-####-########## ,  Type: RAV

grep -iE "<migration-ID>|waiting for maintenance window" app* | wc -l
134469

• On the HCX manager, clearSchedule parameter is set to true which can be found in the log file /common/logs/admin/web.log 

<timestamp> Ent: HybridityAdmin, , TxId: TxId: #######-#####-####-####-#######] INFO  c.v.v.h.a.m.MobilityGroupRestController- Schedule request {"mobilityGroupDefaults":{"transfer":{"startTime":##########,"expiryTime":##########,"isInheritedFromGroupDefaults":true},"switchover":{"startTime":##########,"expiryTime":##########,"clearSchedule":true,"isInheritedFromGroupDefaults":true}}} accepted  for mobility group id : #######-#####-####-####-####### : 

Environment

VMware HCX

Cause

• This issue occurs when the clearSchedule parameter is set to true via API calls.
• This configuration triggers aggressive status updates ("waiting for maintenance window"), which causes the target remoting jobs to be queued up.
• This backlog impacts the communication between the Source HCX Manager and the Target HCX Manager. Consequently, the site-pair status cannot synchronize and remains pending indefinitely.

API:

      "switchoverParams": {
        "switchoverType": "<Migration-Type>",
        "schedule": {
          "startTime": 0,
          "expiryTime": 0,
          "clearSchedule": true
        },

• The API call that can be executed to verify the clearSchedule parameter for a specific Mobility Group is GET https://{{url}}/hybridity/api/v2/mobility-groups/<mobility-group-id>/migrations/<migration-id>

Resolution

• This is a known issue impacting VMware HCX. 

If you believe you have encountered this issue, open a support case with Broadcom Support and refer to this KB article. For more information, see Creating and managing Broadcom support cases.

Additional Information

This issue is fixed in HCX 4.11.4