"Name or service not known" error when Cloud Controller jobs fail when upgrading Tanzu Application Service and using S3/S3-compatible file storage

Products

• VMware Tanzu Application Service for VMs

Issue/Introduction

When you are upgrading Tanzu Application Service (TAS) to v2.9.5+ or v2.10 and higher, and you are using S3/S3-compatible file storage, Cloud Controller (CC) jobs fail to start.

The cloud_controller_ng.log will contain an error similar to:

{"timestamp":"2022-01-25T22:10:49.358413068Z","message":"Error with blobstore: Excon::Error::Socket - getaddrinfo: Name or service not known (SocketError)","log_level":"error","source":"cc.error_handling_client","data":{},"thread_id":66320,"fiber_id":66380,"process_id":9,"file":"/var/vcap/data/packages/cloud_controller_ng/GUID/cloud_controller_ng/lib/cloud_controller/blobstore/xxx/error_handling_client.rb","lineno":67,"method":"rescue in error_handling"}

Environment

Product Version: 2.10

Resolution

This error is caused by the "Path-style S3 URLs (deprecated)" checkbox on the TAS tile's "File Storage" tab being unchecked (default) when the S3-compatible storage supports only path-style URLs. This checkbox is found under "Cloud Controller filesystem" when configuring an "External S3-compatible filestore".

For many S3-compatible file storage systems, path-style S3 URLs should still be used. For more information, refer to AWS S3 Path-Based Access Model.