Harbor Database fails to start with ImagePullBackOff in TMC Self-Managed (Circular Dependency)

Products

• VMware Tanzu Kubernetes Grid Management
• VMware Tanzu Mission Control

Issue/Introduction

 

• Harbor pods in tmc-local namespace are stuck in ImagePullBackOff or ErrImagePull.

• kubectl describe pod indicates the database or core components are trying to pull images from the local Harbor FQDN while Harbor is offline.

• The PackageRepository shows a Reconcile failed status.

 

Environment

TKGM:  2.5.4

TMC: 1.4.2

Cause

Harbor installation is configured to pull from its own internal registry, but a failure in the database or storage prevents Harbor from serving those images.

Resolution

1. Identify the failing Repository:

   kubectl get pkgr -n tmc-local

2. Edit the PackageRepository:

   1. Update the URL to point to the Broadcom public registry instead of the local Harbor FQDN:

      kubectl edit pkgr <harbor-repo-name> -n tmc-local

   2. Change: image: <local-harbor>/tmc/package-repo:v1.x

      To: image: projects.packages.broadcom.com/tmc-self-managed/package-repository:v1.x
(Note: Use the specific version matching your TMC release).

3. Monitor the Database Start:

   1. kubectl get pods -n tmc-local -w

      Once the repository is updated, kapp-controller will pull the manifests and images from Broadcom.

4. Revert to Local:

   1. Once Harbor is healthy and images have been re-verified/pushed to the local registry, edit the PackageRepository again to point back to your internal Harbor URL to restore air-gapped functionality. See Copying Tanzu Standard and Inspection Images