Troubleshooting NSX Upgrade Failures
search cancel

Troubleshooting NSX Upgrade Failures

book

Article ID: 378902

calendar_today

Updated On:

Products

VMware NSX

Issue/Introduction

  • This article provide information on troubleshooting upgrade failures.
  • NSX Upgrade involves the following major component
    • Upgrade Coordinator
    • Prechecks
    • Edge
    • Host
    • Manager
  • Prior to NSX 4.0.1.1 there was a defined upgrade order of Edge, followed by Host, followed by Manager.
  • From 4.0.1.1, some flexibility was added that allows the order of Edge and Host component upgrade to be interchangeable.
  • In all cases, all Edges and all Hosts must be upgraded before the Manager upgrade will start.
  • The NSX main upgrade bundle, .mub, contains the software needed to upgrade all components. It also contains the prechecks.
  • The NSX pre-upgrade checks bundle, .pub, is an asynchronous prechecks bundle decoupled from the main upgrade bundle. This allows Broadcom to dynamically update upgrade prechecks if we become aware of an issue that may impact upgrades.

Resolution

Downloads

  • On the download portal 3.x software is found under product VMware NSX-T, 4.x software is found under product VMware NSX.
  • NSX LE software is a Limited Export edition that does not support IPSec VPN and L2 VPN. Standard licenses will not work for LE software. LE version deployed in error cannot later be converted to standard NSX.

Prereqs

  • Confirm the target NSX version is compatible with other products in the environment. Also confirm the NSX upgrade path is supported. Reference Interop Matrix.
  • Ensure ports required for the upgrade are open e.g. 443/8080 see Required Ports.

For upgrade from NSX 3.2.0, or 3.2.0.1, the upgrade should be initiated from the the orchestrator node.
For upgrade from NSX 3.2.1.x and later, the upgrade can be initiated from any NSX manager node.
To identify the orchestrator node, execute the command "get service install-upgrade" from admin CLI on any one of the NSX Managers, the "Enabled on" field identifies the orchestrator node.


Upgrade Guides:


Log locations:

NSX Manager

    • /var/log/upgrade-coordinator/upgrade-coordinator.log (This log is mostly useful prior to the step shutdown_manager, after that Upgrade Coordinator services will be in stopped state.)
    • /var/log/resume-upgrade.log (This log is only useful after the step shutdown_manager on the orchestrator node.)
    • /var/log/policy/data-migration.log (If the upgrade step has failed in run_migration_tool, then this log is useful to identify the cause of the failure.)
    • /var/log/proton/data-migration.log
    • /var/log/nsx-cli/nsxcli.log (Used to verify the playbook task start and completion status on the orchestrator node.)
    • /var/log/repository/access.log
    • /var/log/syslog

Known issues for upgrade bundles, Upgrade Coordinator and issues initiating the upgrade

For pre-check/post-check, Edge, Host and Manager Known Issues see the component KBs


Handling Log Bundles for offline review with Broadcom support:

Additional Information

Powered by Wolken Software