NSX-T upgrade (3.0.0 -> 3.0.1) via SDDC Manager fails due to known issue within NSX-T

Products

• VMware Cloud Foundation
• VMware NSX Networking

Issue/Introduction

Symptoms:

• NSX-T upgrade (3.0.0 -> 3.0.1) via SDDC Manager fails at "NSX T HOST CLUSTER UPGRADE COODINATOR" phase.
• SDDC Manager UI shows the following:

Description: Check if there are any failures for Host Cluster Upgrade Coordinator Checks
Health Status: RED
Error Description: Failed to execute check for sufficient free space on /tmp partition of all ESXi hosts.
Impact: High: Do not perform upgrade without addressing the issue. NSX-T Upgrade Coordinator Checks Failed with an error.
Remediation: Please remediate the component precheck failures

• In theupgrade-coordinator.log file, you see the entries similar to:

2020-09-18T04:32:00.197Z  INFO pool-21-thread-1 UpgradeServiceImpl - SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="upgrade-coordinator"] Executing check Check for sufficient free space on tmp partition of all ESXi hosts
2020-09-18T04:32:00.197Z  INFO pool-21-thread-1 UpgradeMessagingServiceImpl - SYSTEM [nsx@6876 comp="nsx-manager" level="INFO" subcomp="upgrade-coordinator"] Sending RPC to localhost for 40d07bfc-2d84-484f-b739-594d5b6bd4f6: [com.vmware.nsx.sfhc.SfhcMessage] message_type: MT_RUN_COMMAND
operation_id: "b1dbf8c0-2fd2-44ce-be1a-ce32f350098d"
service_fabric_object {
  fabric_object_identifier: "d089712d-5818-4259-9242-31cc3f199afd"
  runCommand {
    command: "vdf -p | grep tmp | sed \'s/ \\+/ /g\' | awk \'{print $3, $6}\'"
  }
}
return_same_correlationid: true
 
Each ESXi host has additional tmp space.
 
# vdf -p | grep tmp
tmp                       2048    262144      7108         0    255036      8104
amsd-tmp                     0    256000         0         0    256000         0  <=====
sut-tmp                      0    256000      1340         0    254660      1340  <=====

Environment

VMware NSX-T Data Center 2.x
VMware NSX-T Data Center 2.5.x
VMware NSX-T Data Center

Resolution

This issue is fixed in NSX-T 3.0.2 available at VMware Downloads.


Workaround:
1. Upgrade manually via NSX-T Manager first.

1.  Login to NSX-T manager using admin user.
2.  Select [System] -> [Upgrade]
3.  Select [CONTINUE WITH UPGRADE]
4.  [2. Edge] Select [NEXT]
5.  [3. Host] Select [START]

 2. After the upgrade works fine via NSX-T Manager UI, return to SDDC Manager and select [RETRY UPDATE].