Failed to download a Glance image due to networking issues

Products

• VMware Integrated OpenStack

Issue/Introduction

Symptoms:

• Trying to download a glance image from Openstack, this operations is failed with "500 Internal Server Error".
• In glance-api.log, you see entries similar to:

2020-06-02T10:19:20Z 2020-06-02 10:19:20.405 11 WARNING glance.location [req-########-####-####-############ #### #### - #### ####] Get image ########-####-####-########### data failed: Error occurred while opening URL: https://<url>/nfc/########-####-####-###########/disk-0.vmdk for reading.
2020-06-02T10:19:20Z Cause: ("bad handshake: SysCallError(104, 'ECONNRESET')",).: VimException: Error occurred while opening URL: https://<url>/nfc/########-####-####-###########/disk-0.vmdk for reading.
2020-06-02T10:19:20Z 2020-06-02 10:19:20.406 11 ERROR glance.location [req-########-####-####-########### #### #### - #### ####] Glance tried all active locations/stores to get data for image ########-####-####-####-############ but all have failed.: VimException: Error occurred while opening URL: https://<url>/nfc/########-####-####-###########/disk-0.vmdk for reading.

Environment

7.x

Cause

This issue is caused by the communication issue between glance-api and ESXi vmkernel port because NFC communication is used for downloading a glance image.

For example, if MTU of vmkernel port is set to "9,000", glance-api and vmkernel port tries to use "MSS=8960" for TCP communication. However, MTU configurations of Openstack controller nodes are "1,500" by default. As a result, packet with larger size than 1500 are dropped at vNIC of an Openstack controller node running glance-api.

Resolution

This issue can be fixed by troubleshooting network issues.
For example, this issue can be fixed by matching MTU among vmkernel port and Openstack controller nodes.