Unable to pull images using tags after upgrading Harbor

Products

• VMware Tanzu Kubernetes Grid

Issue/Introduction

Symptoms:

Pulling images from Harbor using tags fails with error:

docker pull harbor.vmware.com/repository1/image-test:1.0.0
Error response from daemon: unknown: artifact repository1/image-test:1.0.0 not found

The image pull works when using the sha256:

docker pull harbor.vmware.com/repository1/image-test:1.0.0@sha256:65e3b69cba34b84fe95fc1dd00ec631cdb6b57186874ad8618c96d90272a66b3

 

Similar messages can be found on the harbor-app logs:

level=error msg="Not continuing with pull after error: unknown: artifact repository1/image-test:1.0.0 not found"

 

Environment

VMware Tanzu Kubernetes Grid Integrated Edition 1.x

Cause

During the upgrade, the unique index on the repository table got corrupted and this created duplicate entries. 

Resolution

Connect to the Harbor Database:

docker exec -it harbor-db bash
psql -U postgres -d registry
 

1. Find the duplicated repository name:

select * from (select name, count(*) cnt from repository group by name) a where a.cnt > 1;
 

2. Find the ID of the duplicated repository and get the new_id and old_id:

select repository_id, name, creation_time from repository where name like ‘%prometheus/test-job%’ ;
 

3. Update the tag and artifact to point to the new repository id:

update artifact set repository_id = <new_id> where repository_id = <old_id>;

update tag set repository_id = <new_id> where repository_id = <old_id>;

delete from repository where repository_id = <old_id>;
 

After all the duplicated repositories have been removed, no row should be returned when running the following query:

select * from (select name, count(*) cnt from repository group by name) a where a.cnt > 1
 

4. Reindex the table

REINDEX TABLE repository;
 

5. Verify if the image can be pulled.