Exporting a VM as OVF template to content library or deploying an VM from OVF library item results in timeout error and export/import fails

Products

• VMware vCenter Server

Issue/Introduction

Symptoms:

1. Deployment of an OVF/OVA template from content library fails with "vim.fault.Timedout" error.

• Timeout error is seen in vpxd.log

-- testpyves0 -- ResourcePool.ImportVAppLRO: vim.fault.Timedout:
--> Result:
--> (vim.fault.Timedout) {
-->    faultCause = (vmodl.MethodFault) null, 
-->    faultMessage = <unset>
-->    msg = ""
--> }
--> Args:
-->

• Error is seen in cls.log

  ImportSessionActivity          | OVF import session 330b37b4-c297-4b91-aeea-da54a52f78bb failed with internal error.
  (vim.fault.Timedout) {
     faultCause = null,
     faultMessage = null
  }

2. Importing a vAPP/VM to content lirary fails with  "vim.fault.Timedout" error.

• Timeout error is seen in vpxd.log

  --> vt_win_2019_202106091610_jk24_fc -- VirtualMachine.ExportVmLRO: vim.fault.Timedout:
  --> Result:
  --> (vim.fault.Timedout) {
  -->  faultCause = (vmodl.MethodFault) null,
  -->  faultMessage = <unset>
  -->  msg = ""
  --> }
  --> Args:

• Timeout error in cls.log

  ExportSessionActivity          | OVF export session 9fa3d620-75dc-4d3f-9461-11cee7daef96 terminated abnormally.
  (vim.fault.Timedout) {
     faultCause = null,
     faultMessage = null

   

Environment

VMware vCenter Server 7.0.3
VMware vCenter Server 7.0.2

Cause

Long running operations by Content Library, which require usage of HttpNfcLease sessions, are encountering cleaned up sessions.

Content Library requires the use of HttpNfcLease in certain operations, like exporting a Virtual Machine to Content Library, or while deploying a Content Library managed OVF item. However, since the actual data transfer occurs out of band and not on the HttpNfcLease channel, no actual data transfer is detected on the HttpNfcLease channel, thus causing the channel to expire and get cleaned up, which in turn causes timeout on the vCenter operation.
By default vCenter Server renews the HttpNfcLease session on Vpxa every 20 minutes.
If the transfers involving Content Library take more than the default 20 minutes, the operation would encounter this timeout error.

Resolution

Note: These steps are only available in vCenter Server 7.0 Update 3f or newer versions. If you are using an older version of vCenter Server, please update to 7.0 Update 3f (or newer), before following the steps below.
Units for httpNfcTicket.renewalPeriod are now in seconds. Originally they were in Micro-seconds.

In order to avoid these timeout errors, you can extend the timeout by updating the following vpxd properties:

Refer: VMware vCenter Server 7.0 Update 3d Release Notes
 

Setting	Description
httpNfcTicket.renewalPeriod	Allows setting the length of the httpNFClease renewal period in seconds. At the end of this period, vpxa will check if there has been any progress in transferring files related to the httpNFClease. If there was progress, the lease's ticket will be renewed. If there wasn't, the lease's ticket will be expired. The standard renewal period is 20 minutes.
httpNfcTicket.timeoutPeriodMinute	This is NFC ticket expiration period in minutes, which by default is 30 minutes on Vpxa. Vpxa is where tickets are maintained/registered and optionally created. Vpxd creates the ticket and registers it on Vpxa through ticket manager. The lease(ticket) timeout period in Vpxa needs to be configurable in vpxd config option so the user can change it based on application and also they can both be in sync.

 

"UI also provides an option to set these configs. To set the both options via UI, apply the following steps:

Click on VC on left side UI-> configure -> Advanced Settings -> Edit settings ->
find config.httpNfcTicket.timeoutPeriodMinute-> set to 60
find config.httpNfcTicket.renewalPeriod -> set to 3600
Click Save"

For example, to set the both options to 60 minutes, apply the following steps:

1. Connect to VCSA using SSH session with root

2. Take a backup of the file /etc/vmware-vpx/vpxd.cfg

   # cp /etc/vmware-vpx/vpxd.cfg /etc/vmware-vpx/vpxd.old

3. Confirm vpxd is not running / Stop vpxd using:

   # service-control --stop vmware-vpxd

4. Edit vpxd.cfg

# vi vpxd.cfg

5. Add the following entries to the file inside the <config> tag:

   <config>
      <httpNfcTicket>
         <timeoutPeriodMinute>60</timeoutPeriodMinute>
         <renewalPeriod>3600</renewalPeriod>
      </httpNfcTicket>

Note: httpNfcTicket is directly under <config> section as shown above, in the config file.
This section is not under <vpxd> or other sections.

6. Save the file and restart vpxd using:

   # service-control --start vmware-vpxd

Note: The values should not be too long to avoid having unnecessary sessions around.