VIO task failure when a vSphere operation run for a prolonged period of time

Products

• VMware
• VMware Integrated OpenStack

Issue/Introduction

Increasing the Keystone token expiration time resolves the issue.

Symptoms:
When running a task in VMware Integrated Openstack (VIO), if the operation on vSphere run for a prolonged period of time and takes more than 1 hour by default the task fails on the Openstack.

Environment

VMware Integrated OpenStack 2.0.x
VMware Integrated OpenStack 2.5.x
VMware Integrated OpenStack 3.0.x

Cause

This issue occurs when the expired Keystone token is used.

Resolution

Each task in Openstack must be authenticated to interact with other components of Openstack. To achieve this components must request an authentication token from keystone. This token is used for communicating with various Openstack components such as nova, neutron, and cinder.

However, this token has an expiration time period (by default it is one hour). When a task does not complete within that given time frame, the first operation of the task after the expiration date fails due to an invalid token (caused due to the expiration time period).

Note: Review the task and validate if it can be run differently to avoid exceeding the expiration time or renew the token before the failure occurs. However, if this option is not available, increase the expiration time of the Keystone token.

To resolve this issue, increase the Keystone token expiration time:

1. Connect the open-stack management server (OMS) using SSH as a vio user.
2. Locate the the /opt/vmware/vio/custom/custom.yml file.
   
   If the custom.yml does not exist, you must create it.
   
   To create the /opt/vmware/vio/custom/custom.yml:
   1. Run this command:
      
      cp /var/lib/vio/ansible/custom/custom.yml.sample /opt/vmware/vio/custom/custom.yml
       
   2. Type sudo -i and enter the a vio user password.
       
3. Type vi /opt/vmware/vio/custom/custom.yml.
4. Go to the Keystone section.
5. Add/Update the following entry:
   
   keystone_token_expiration_time: new_value_in_seconds
    
6. Click Save.
7. Type exit.
8. Type exit.

NOTE: This value is missing in the sample file for VMware Integrated Openstack 4.0 .  Manually entering the entry from step 5 will work