How to increase max_job_workers in Harbor Jobservice

Products

• Tanzu Kubernetes Runtime

Issue/Introduction

Harbor Jobservice container supports "max_job_workers" parameter.  Harbor Official Document defined "max_job_workers" as follows.

"The maximum number of replication workers in the job service. For each image replication job, a worker synchronizes all tags of a repository to the remote destination. Increasing this number allows more concurrent replication jobs in the system. However, since each worker consumes a certain amount of network/CPU/IO resources, set the value of this attribute based on the hardware resource of the host. The default is 10."

However, Tanzu Standard Package Harbor doesn't support "max_job_workers" as a configurable parameter.

Environment

Harbor v2.9.1 from Tanzu Standard Package

Resolution

1. Make the Harbor Registry Read Only mode

Harbor WebUI --> Administration --> Configuration --> System Settings --> Check "Repository Read Only" --> SAVE
Reference - Harbor Official Document

2. Generate a YTT Overlay YAML file - Increase workers from 10 to 20.

cat > add-max-job-workers-overlay.yaml <<EOF
#@ load("@ytt:overlay", "overlay")
#@ load("@ytt:data", "data")
#@ load("@ytt:yaml", "yaml")

#@ def patch_max_job_workers(old, _):
#@   config = yaml.decode(old)
#@   config.setdefault("worker_pool", {})["workers"] = 20
#@   return yaml.encode(config)
#@ end

#@overlay/match by=overlay.and_op(overlay.subset({"kind": "ConfigMap"}), overlay.subset({"metadata": {"name": "harbor-jobservice"}}))

---
data:
  #@overlay/replace via=patch_max_job_workers
  config.yml:
EOF

 

3. Update the Harbor package

kubectl get pkgi -A | grep -E 'NAME|harbor'
NAMESPACE    NAME      PACKAGE NAME               PACKAGE VERSION         DESCRIPTION           AGE
mypackage    harbor    harbor.tanzu.vmware.com    2.9.1+vmware.1-tkg.1    Reconcile succeeded   6d22h
pkg_namespace=mypackage

# Update the package
tanzu -n ${pkg_namespace} package installed update harbor --ytt-overlay-file add-max-job-workers-overlay.yaml
...
3:38:58PM: Deploy succeeded

 

4. Verify the result

POD=$(kubectl -n tanzu-system-registry get pod -l app=harbor,component=jobservice -o jsonpath='{.items[0].metadata.name}')
kubectl -n tanzu-system-registry exec $POD -- cat /etc/jobservice/config.yml | yq .worker_pool.workers
20

 

5. Revert the Harbor registry READONLY mode

Harbor WebUI --> Administration --> Configuration --> System Settings --> Uncheck "Repository Read Only" --> SAVE