cci-ns-controller-manager pods fail to pull image with Http request failed Code 400 ErrorType(2)

Products

• VMware vCloud Suite

Issue/Introduction

During the deployment of a new VMware Cloud Foundation (VCF) 9.1 environment, control plane creation fails to complete. Multiple cci-ns-controller-manager pods remain stuck in a deploying state.

Symptoms and Errors:

• cci-ns-controller-manager pods are unable to pull container images from the repository.

• The pod deployment logs report the following error: Http request failed. Code 400. ErrorType(2)

Environment

VMware Cloud Foundation (VCF) 9.1

Cause

A custom external DNS record exists for the docker-registry FQDN, docker-registry.kube-system.svc, pointing to an invalid or incorrect IP. This overrides internal DNS resolution, causing the pod to send image pull requests to itself and resulting in HTTP 400 errors.

Resolution

1. Access the local or external DNS server servicing the VCF 9.1 deployment environment.

2. Locate and delete the static/custom DNS record mapping docker-registry.kube-system.svc to the pod IP address.

3. Restart or allow the cci-ns-controller-manager pods to redeploy and pull container images.