Edge upgrade fails with error power off task failed on edge TransportNode
Article ID: 393569
Updated On:
Products
Issue/Introduction
- During NSX upgrade of Edge nodes upgrade fails on an edge node with the error below:
Edge <Target upgrade version of edge>/Edge/nubVMware-NSX-edge-<Target upgrade version of edge>.nub power off task failed on edge TransportNode <TN UUID> clientType EDGE. target edge fabric node id <TN UUID>. return status null - Edge VM will be in NSX maintenance mode and the upgrade status will show the switch OS step is complete.
- vCenter will show a power off guest OS task for the Edge.
- Edge VM web console may show a failed Power Off task when attempting to login as below:
Environment
VMware NSX
VMware NSX-T Data Center
Cause
This issue is caused by the Edge failing to power off after being given the power off guest OS command. The task will show complete, meaning the command has been sent to the VM however the VM has not powered off. This is part of the normal upgrade process as detailed in the admin guide (Edge upgrade workflow). Once the OS has been upgraded, the VM should be powered off and then powered back on automatically by the NSX upgrade coordinator. However in some circumstances the VM OS fails to power off fully and will time out the task leading to the upgrade workflow failing.
Resolution
Connecting to the Edge VM web console may show what appears to be a normal console, attempting to login (by hitting enter to get console) may show the above CLI error for power off timeout. The VM will then forcibly power down. In such a case, the VM can be powered back on in vCenter and the upgrade retried as per the normal upgrade workflow.
Alternatively the VM can be manually restarted once the upgrade has been paused in a failed state and the upgrade retried via the GUI.
Additional Information
If you are facing this issue and the above work arounds do not allow the upgrade to complete then please open a ticket with Broadcom support about this issue, please provide the following:
- NSX Manager support bundles.
- NSX Edge support bundles.
- Text of any error messages seen in NSX GUI upgrade windows.
Handling Log Bundles for offline review with Broadcom support
Feedback
