"Failed migrations: Migrating VrfPrefixListMigrationTask" NSX upgrade failure.

Products

• VMware NSX

Issue/Introduction

• NSX Manager upgrade dry run failed. Do not proceed with the upgrade. Please collect the support bundle and contact VMWare GS. Failed migrations: Migrating VrfPrefixListMigrationTask. VrfPrefixListMigrationTask: NullPointerException: Exception during Data migration
• Upgrade pre-check fails with the error "Failed migrations: Migrating VrfPrefixListMigrationTask. VrfPrefixListMigrationTask: NullPointerException: Exception during Data migration". 
• From the Logical Migration logs, var/log/upgrade-coordinator/logical-migration.log, you'll see an error similar to below example:
  
  <DATE>T01:59:26.396Z  INFO pool-12-thread-1 VrfPrefixListMigrationTask 27930 - [nsx@6876 comp="nsx-manager" level="INFO" subcomp="manager"] Prefix list Id of type T0_INTERNAL_TRANSIT #####-####-####-####-####### for VRF id <UUID of the problem VRF>
<DATE>T01:59:26.396Z  WARN pool-12-thread-1 UfoCorfuTableMigrator 27930 - [nsx@6876 comp="nsx-manager" level="WARNING" subcomp="manager"] ERROR while running logical migration MappingDetails{name='VrfPrefixListMigrationTask', runForTheseFromVersions='3.2,4.0', targetVersion='4.1', modelName='null', migrationType=null, reason='This migration task will run for migrations from GC/HL/3.2.0 to 4.1. It will add missing prefix entries to VRF.', customMigratorClassName='com.vmware.nsx.management.migration.impl.VrfPrefixListMigrationTask', fieldMappings=null, targetProtoName='null', requiresCustomCode='false', owner='null', apiToTest='null', weight='1.0', impactedTables='InternalPrefixListConfig', cleanup='false', enabled'false', diskBacked'false', triggerMode'null', supportsResolve'false', component'null'}

Environment

VMware NSX

Cause

When a VRF prefix entry does not exist in the 'PrefixListConfig' table, and does exist within 'LogicalRouter', 'GPRR' and 'PrefixList table', the upgrade pre-check will fail. 

Resolution

This is a known issue impacting VMware NSX.

The following are the workaround options: 

Option 1: 

1. The simplest workaround, for the VRF ID to which the pre-check is failing against, is to remove the VRF, if no devices are using it. 
2. Re-try the upgrade. 

Or, 

Option 2: 

1) Take a standard NSX backup and ensure the passphrase is known. 
2) Download the .jar file attached to this KB.
3)Through WinSCP, copy the .jar file and transfer to an NSX Manager (any will do).  
4) SSH to NSX manager as root user, proceed to where the .jar file location was saved to, and enter the below: 
# java -cp "libcorfu-data-corrector-lib.jar:/opt/vmware/proton-tomcat/webapps/nsxapi/WEB-INF/lib/*:/usr/tomcat/lib/*" -Dcorfu-property-file-path=/opt/vmware/proton-tomcat/conf/ufo-factory.properties -Djava.io.tmpdir=/image/corfu-tools/temp com.vmware.nsx.management.migration.impl.PrefixListDataCorrector <username> <password> 
- Username and password is the same for NSX manager login. 
5) Once run, no output will mean a successful run.
6) A dummy update is now needed to the VRF to finalize the remediation.
On the NSX UI, go to Networking > Tier-0 Gateway > edit settings of the VRF, add any text to the Description box and click Save: 

7) Proceed with the NSX upgrade. 

To note, this script can be run on NSX 3.2 / 4.1 and 4.2 versions.