NSX-T Manager upgrade precheck fails for /nonconfig partition

Products

• VMware NSX

Issue/Introduction

Symptoms:

• NSX-T Data Center
• Previously the environment was upgraded from a version of NSX prior to 3.0 to 3.0.x
• NSX UI Upgrade Prechecks reports the error

/nonconfig partition not mounted on <NSX Manager>. NSX Manager cannot be upgraded.

• ssh to the Manager confirms nonconfig parition is not present

As admin
> get filesystem-stats
nonconfig is not listed in the output
or
As root
/# mount -l /nonconfig
mount: nonconfig: can't find in /etc/fstab.

Environment

VMware NSX-T Data Center 3.x
VMware NSX-T Data Center

Cause

NSX-T Managers upgraded from versions prior to 3.0 to 3.0.x and above required a 100GB disk to be added to the NSX Manager.
This was partitioned and mounted as /nonconfig

> get filesystem-stats | find nonconfig
/dev/mapper/nsx-secondary     98G  679M   93G   1% /nonconfig

Due to a software issue on NSX-T 3.0.x, if a CDROM was attached to the NSX Manager at the time of upgrade, the /nonconfig partition was not mounted as expected however the upgrade succeeded.
VMware added an upgrade precheck to detect this condition as any future upgrades may fail without a nonconfig partition.
This issue was resolved in NSX-T 3.1.0 so Managers upgraded directly from pre 3.x to 3.1.0 or above with an ISO attached will not be impacted.

Resolution

This issue is resolved in NSX-T Data Center 3.1.0 and above.

Workaround:

Any Manager node which does not have a nonconfig partition will need to be replaced.

If the NSX Manager orchestrating the upgrade is not impacted then the Manager node(s) impacted should be replaced one at a time.
Rerun the prechecks and once they succeed, proceed with the upgrade.

If the NSX Manager orchestrating the upgrade is impacted, please follow these recovery steps.

1) ssh to the orchestrator NSX Manager as admin user
2) Stop the upgrade process
    > stop service install-upgrade  
3) Elevate to root with 'st en' and delete the Upgrade Plan via API call
    curl -v -k -u admin -H "Content-Type:application/json" -X DELETE https://NSX_MGR/api/v1/upgrade-mgmt/plan
4) Replace the orchestrator node
5) Upload the upgrade bundle again via the UI
6) Rerun prechecks and proceed with upgrade