NSX Managers Upgrade from 3.2 to 4.2 fails with Error: Found data inconsistencies: ERROR: GroupEnitityBarrierEntityTypeCleanupTask: Stale Group to NsGroup entities

Products

• VMware NSX

Issue/Introduction

Errors below seen when running NSX Managers upgrade from NSX versions 3.2.x to 4.2.x and above:

• "NSX Managers upgrade has failed, check error details to determine if manual resolution is needed and 'Retry Upgrade'."
• "Upgrade has failed, Try to fix the issues and 'Retry Upgrade' else connect to GS to initiate rollback."
• "Unexpected error while upgrading upgrade Unit. Found data inconsistencies: ERROR: GroupEnitityBarrierEntityTypeCleanupTask: Stale Group to NsGroup entities ####"

Note: #### represents the number of stale entries found and need to be resolved.

Additional upgrade issues observed:

• The pre-check error may not be seen when running the upgrade using SDDC or another method other than with the NSX UI.  The NSX managers will fail to upgrade.
• Retrying the upgrade in the NSX UI at the NSX Managers stage will fail.
• Error seen from NSX Manager UI under System, Upgrade, NSX Managers will not have a resolve option.

Environment

VMware NSX 4.2.x

Cause

• The stale entries are not an issue prior to the upgrade, and this is expected behavior when stale entries are found.
• With no remediation, the stale "NsGroup" entries can be problematic if remaining after an upgrade to 4.2.x and above.
• The cleanup task "GroupEnitityBarrierEntityTypeCleanupTask" is available to resolve the stale entries.
• This cleanup tool was introduced in 4.2.0 and should be included in all future releases.  
• The stale entries found will need to be resolved in order for the upgrade of the NSX Managers to continue.

Resolution

1. In the NSX UI under System, Upgrade, Check Upgrade Readiness, the prechecks will need to be run again for the NSX Managers. 
2. Once the NSX Manager precheck completes, then the error should be shown with the action to click on "RESOLVE" 
3. Clicking on the action "RESOLVE" should address and remove the number of stale entries found. 
4. Wait for the GroupEnitityBarrierEntityTypeCleanupTask operation to complete. This operation will take a few minutes to complete.
5. The NSX Managers should now be able to proceed with the regular upgrade process.

Additional Information

• These stale entries can be present in NSX 3.2.x and there is no relation to any data corruption in Corfu.
• The cleanup process to resolve the stale NsGroup entries has been merged into NSX version 4.2.x