Metadata connection fails with HTTP code 403 for tenant instances in VMware Integrated OpenStack 7.x

Products

• VMware
• VMware Integrated OpenStack

Issue/Introduction

Symptoms:

This is for VMware Integrated OpenStack deployments with NSX-T. When tenant instances attempt to reach the metadata service for data the instances will connect but receive a HTTP error 403 in return.

Environment

VMware Integrated OpenStack 6.x
VMware Integrated Openstack 7.x

Cause

The metadata proxy secret is incorrect between NSX-T and OpenStack.

Resolution

This is due to the metadata shared secret being incorrect between VMware Integrated OpenStack (VIO) and NSX-T. This will need to be corrected with the following steps:

 

1. Ensure the secret is correct in the metadata proxy in NSX-T. A new secret may be set within the metadata proxy before the following steps.
2. SSH to the VIO manager as root.
3. Encode the shared secret with the following command and save the output: echo -n 'your-shared-secret' |base64
4. Edit the managedpasswords secret and update the metadata_proxy_shared_secret row with the output from step #3. To edit the secret run this command: osctl edit secrets managedpasswords
5. Edit the managedencryptedpasswords secret. Save the row for metadata_proxy_shared_secret externally before deleting the entire row. To edit the secret run this command: osctl edit secrets managedencryptedpasswords
6. Wait for the nova and neutron pods to restart. We can watch viocli get deployment or watch pods for changes. After a few minutes we will no longer receive the 403 error for metadata.

Additional Information

Impact/Risks:
The resolution will restart nova and neutron containers. There will be a temporary interruption to services.