Failed to download a Glance image due to networking issues
Article ID: 321759
Updated On:
Products
VMware
VMware Integrated OpenStack
Issue/Introduction
Symptoms:
2020-06-02T10:19:20Z Cause: ("bad handshake: SysCallError(104, 'ECONNRESET')",).: VimException: Error occurred while opening URL: https://ctaosk2a-phpv001.cta.kddi.local/nfc/52be56f0-5770-6fa7-c498-ad2d065de023/disk-0.vmdk for reading.
2020-06-02T10:19:20Z 2020-06-02 10:19:20.406 11 ERROR glance.location [req-03e308e7-0ced-4be4-bad6-df80a607a8d9 fdcfb08df7287cf91e101fb7e639bde86642089467b0385cf8e85e222c2fff1c 72e26b8eb1d745fcae12654740c2c77a - 73d1117d661241ea98c8c2f33608a7e1 73d1117d661241ea98c8c2f33608a7e1] Glance tried all active locations/stores to get data for image 59ba5285-3726-44be-9767-58f25bdac915 but all have failed.: VimException: Error occurred while opening URL: https://ESXi.acme.local/nfc/52be56f0-5770-6fa7-c498-ad2d065de023/disk-0.vmdk for reading.
- Trying to download a glance image from Openstack, this operations is failed with "500 Internal Server Error".
- In glance-api.log, you see entries similar to:
2020-06-02T10:19:20Z Cause: ("bad handshake: SysCallError(104, 'ECONNRESET')",).: VimException: Error occurred while opening URL: https://ctaosk2a-phpv001.cta.kddi.local/nfc/52be56f0-5770-6fa7-c498-ad2d065de023/disk-0.vmdk for reading.
2020-06-02T10:19:20Z 2020-06-02 10:19:20.406 11 ERROR glance.location [req-03e308e7-0ced-4be4-bad6-df80a607a8d9 fdcfb08df7287cf91e101fb7e639bde86642089467b0385cf8e85e222c2fff1c 72e26b8eb1d745fcae12654740c2c77a - 73d1117d661241ea98c8c2f33608a7e1 73d1117d661241ea98c8c2f33608a7e1] Glance tried all active locations/stores to get data for image 59ba5285-3726-44be-9767-58f25bdac915 but all have failed.: VimException: Error occurred while opening URL: https://ESXi.acme.local/nfc/52be56f0-5770-6fa7-c498-ad2d065de023/disk-0.vmdk for reading.
Environment
VMware Integrated Openstack 7.x
Cause
This issue is caused by the communication issue between glance-api and ESXi vmkernel port because NFC communication is used for downloading a glance image.
For example, if MTU of vmkernel port is set to "9,000", glance-api and vmkernel port tries to use "MSS=8960" for TCP communition. However, MTU configurations of Openstack controller nodes are "1,500" by default. As a result, packet with larger size than 1500 are dropped at vNIC of an Openstack controller node running glance-api.
For example, if MTU of vmkernel port is set to "9,000", glance-api and vmkernel port tries to use "MSS=8960" for TCP communition. However, MTU configurations of Openstack controller nodes are "1,500" by default. As a result, packet with larger size than 1500 are dropped at vNIC of an Openstack controller node running glance-api.
Resolution
This issue can be fixed by troubleshooting network issues.
For example, this issue can be fixed by matching MTU among vmkernel port and Openstack contoller nodes.
For example, this issue can be fixed by matching MTU among vmkernel port and Openstack contoller nodes.
Feedback
Yes
No
Powered by
