VMware NSX-T upgrade from 3.0.0 to 3.0.1 via SDDC Manager fails due to known issue within NSX-T

Products

• VMware Cloud Foundation
• VMware NSX

Issue/Introduction

• NSX-T upgrade (3.0.0 -> 3.0.1) via SDDC Manager fails at "NSX T HOST CLUSTER UPGRADE COODINATOR" phase.
• SDDC Manager UI shows the following:

Description: Check if there are any failures for Host Cluster Upgrade Coordinator Checks
Health Status: RED
Error Description: Failed to execute check for sufficient free space on /tmp partition of all ESXi hosts.
Impact: High: Do not perform upgrade without addressing the issue. NSX-T Upgrade Coordinator Checks Failed with an error.
Remediation: Please remediate the component precheck failures

• In theupgrade-coordinator.log file, you see the entries similar to:

2020-09-18T04:32:00.197Z  INFO pool-21-thread-1 UpgradeServiceImpl - SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="upgrade-coordinator"] Executing check Check for sufficient free space on tmp partition of all ESXi hosts
2020-09-18T04:32:00.197Z  INFO pool-21-thread-1 UpgradeMessagingServiceImpl - SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="upgrade-coordinator"] Sending RPC to localhost for ########-####-####-####-##########f6: [com.vmware.nsx.sfhc.SfhcMessage] message_type: MT_RUN_COMMAND
operation_id: "<uuid>"
service_fabric_object {
  fabric_object_identifier: "<uuid>"
  runCommand {
    command: "vdf -p | grep tmp | sed \'s/ \\+/ /g\' | awk \'{print $3, $6}\'"
  }
}
return_same_correlationid: true
 
Each ESXi host has additional tmp space.
 
# vdf -p | grep tmp
tmp                       2048    262144      7108         0    255036      8104
amsd-tmp                     0    256000         0         0    256000         0  <=====
sut-tmp                      0    256000      1340         0    254660      1340  <=====

Environment

VMware NSX-T Data Center 2.x
VMware NSX-T Data Center 2.5.x
VMware NSX-T Data Center

Resolution

This issue is fixed in NSX-T 3.0.2 available at Support Documents and Downloads (broadcom.com).


Workaround:
1. Upgrade manually via NSX-T Manager first.

1.  Login to NSX-T manager using admin user.
2.  Select [System] -> [Upgrade]
3.  Select [CONTINUE WITH UPGRADE]
4.  [2. Edge] Select [NEXT]
5.  [3. Host] Select [START]

 2. After the upgrade works fine via NSX-T Manager UI, return to SDDC Manager and select [RETRY UPDATE].