NSX 3.2.x Manager upgrade stuck "In Progress" migrating LogicalPortServiceConfig

Products

• VMware NSX

Issue/Introduction

• NSX Manager upgrade to 3.2.x is stuck "In Progress".
• From the admin shell of the NSX Manager orchestrating the upgrade, LogicalPortServiceConfig is stuck at 0%, for example:
  

  Manager>  get upgrade progress-status
  ****************************************************************************
  Node Upgrade has been started. Please do not make any changes, until
  the upgrade operation is complete. Run "get upgrade progress-status"
  to show the progress of last upgrade step.
  ****************************************************************************
  
  Upgrade info:
  From-version: 3.1.3.7.0.19380482
  To-version: 3.2.3.1.0.22104638
  
  Upgrade steps:
  download_os [<DATE> 09:18:29 - <DATE> 09:18:49] SUCCESS
  shutdown_manager [<DATE> 09:18:55 - <DATE> 09:21:15] SUCCESS
  install_os [<DATE> 09:21:15 - <DATE> 09:22:22] SUCCESS
  migrate_manager_config [<DATE> 09:22:24 - <DATE> 09:22:29] SUCCESS
  switch_os [<DATE> 09:22:29 - <DATE> 09:22:35] SUCCESS
  reboot [<DATE> 09:22:35 - <DATE> 12:23:13] SUCCESS
  run_migration_tool [<DATE> 12:24:27 - ] IN_PROGRESS
  ------ Output of last step start ------
  <DATE> 12:24:27.968054 Deleting datastore files
  <DATE> 12:24:27.974518 Copying old datastore files
  <DATE> 12:24:31.348378 Done copying old datastore files
  <DATE> 12:24:33.105740 Start Corfu server
  <DATE> 12:24:37.509382 Process corfu-server started
  
  Overall Progress: (3/6)
  ---- (1) CCP: Completed [1 object(s)] (<DATE> 12:24:37 - <DATE> 12:25:03) ----
  --------------------------------------------------------------------------------------------
  ---- (2) Proton: Completed [564009 object(s)] (<DATE> 09:25:52 - <DATE> 09:28:30) ----
  ---- (3) Policy: Completed [235987 object(s)] (<DATE> 09:28:52 - <DATE> 09:31:51) ----
  ---- (4) Logical: 0% [0 of 305758 object(s)] (<DATE> 09:32:08 - ) ----
  Currently Migrating: LogicalPortServiceConfig 0% [0 of 31717 objects] (<DATE> 09:32:24 - )
  ---- (5) CBM: Pending
  ----------------------------------------
  ---- (6) UFO Checkpointing: Pending

 

Environment

VMware NSX-T 3.2.x

Cause

In environments with a large number of logical ports, the data-migration task for LogicalPortServiceConfig runs extremely slowly due to excessive memory consumption caused by high DB read/write activity.

Resolution

This issue is resolved in VMware NSX-T Data Center 3.2.4, available at Broadcom downloads.

If you are having difficulty finding and downloading software, please review the Download Broadcom products and software KB.

If you believe you have encountered this issue, please open a support case with Broadcom Support and refer to this KB article.

For more information, see Creating and managing Broadcom support cases.

Additional Information

Troubleshooting NSX Manager Upgrade Failures