Limitations of rolling upgrade for NSX.

Products

• VMware NSX

Issue/Introduction

• Prior to NSX 3.2.1 release, all the NSX manager nodes are upgraded in the management cluster simultaneously. The advantage of the parallel upgrade is that it takes less time to upgrade the management plane. The tradeoff is that the management plane will not be available for a period during the upgrade process. With the rolling upgrade, the manager nodes will be upgraded sequentially. During the management upgrade process, there will be at least 2 manager nodes available all the time to make sure that the management plane is always up.
  
  
• Rolling upgrade is only applicable for upgrading from NSX 3.2 to a later version. Do not use rolling upgrade for upgrading from NSX 3.1 to 3.2 or later.

Environment

VMware NSX-T Data Center 3.x
VMware NSX 4.x

Cause

• The UA upgrade playbooks are derived from the 'target' version MUB. Since the same MUB can be used for upgrading from 3.1 to 3.2 or later (non-rolling) and from 3.2 to later (rolling), both playbooks must be provided, and UC will automatically select the appropriate one.
  
  
• Using rolling upgrade for upgrading from 3.1 to 3.2 or later may cause Corfu database corruption, and rolling back the upgrade will not recover from this issue.
  This issue only occurs when 'start upgrade-bundle ...' is manually invoked, so admin can avoid this problem by refraining from using 'start upgrade-bundle ...'.

 

Resolution

1. To upgrade the NSX Manager from 3.1 to 3.2 or later, it should be done through the UI, not via the command line. If the upgrade fails through the UI, 'start upgrade-bundle ...' should not be used. Instead, 'resume upgrade-bundle ...'  should be used to resume the upgrade. 'resume upgrade-bundle ...' will choose correct upgrade mode automatically. The resume-upgrade approach is for non-rolling upgrade where we invoke the entire playbook instead of an individual step.
   
   
2. Restore NSX Manager from a backup if a rolling upgrade was invoked for upgrading from 3.1 to 3.2 or later.
   Refer Restore NSX Manager from Backup