Replicating images from remote Azure Container Registry (ACR) to Harbor fails with 401 Unauthorized error

Products

• VMware Tanzu Kubernetes Grid Integrated Edition

Issue/Introduction

For configuring replication in Harbor, the high-level steps are as follows:
 

• Create a Replication endpoint
• Create a Replication rule
• Run the Replication rule manually

In your current setup:
 

• The replication endpoint is configured to use Azure's ACR as the provider
• The access key and the secret of a user other than the admin user have been supplied to the replication endpoint configuration
• You have successfully tested the connection
• You have created a pull-based replication rule that:
  • Uses Azure's ACR using the replication endpoint configured previously
  • Has the trigger mode set to manual
• Upon running the replication rule manually, you see an error similar to the following (for a Bosh deployed Harbor registry, this can also be seen under /var/vcap/sys/log/harbor/harbor-app-logs/core.log):

May 20 16:41:46 172.20.1.1 core[31537]: 2021-05-20T16:41:46Z [ERROR] [/replication/adapter/native/adapter.go:126]: failed to ping registry https://<registry-name-redacted>: http status code: 401, body: {"errors":[{"code":"UNAUTHORIZED","message":"authentication required, visit https://aka.ms/acr/authorization for more information."}]}
May 20 16:45:06 172.20.1.1 core[31537]: 2021-05-20T16:45:06Z [ERROR] [/replication/operation/controller.go:103]: the execution 5 failed: failed to fetch artifacts: failed to fetch artifacts: failed to list artifacts of repository <repository-name-redacted>: http status code: 401, body: {"errors":[{"code":"UNAUTHORIZED","message":"authentication required, visit https://aka.ms/acr/authorization for more information.","detail":[{"Type":"repository","Name":"redacted","Action":"metadata_read"}]}]}

Although the error message may seem to hint that authentication is the issue, the real issue here is one caused by authorization and more specifically, missing permissions. Read the resolution section to find out more about how to isolate this problem.

Environment

Product Version: 1.9

Resolution

How do I isolate this problem?
 

• If you can access admin credentials for the Azure ACR, you can reconfigure the replication endpoint to use those credentials, then test that the connection still gets success, and finally manually re-run the replication rule. If you are able to pull the images using the admin credentials, then you have evidence to review the permissions of the user that was initially setup.

For reference, to review permissions in Azure ACR, read the following document:

• https://aka.ms/acr/authorization