Harbor replication is failing with harbor database pod showing error "InsufficientFreeMemory"

Products

• VMware vSphere Kubernetes Service

Issue/Introduction

Replication of Harbor as Supervisor service in version 2.13.1 is failing with error InsufficientFreeMemory for the Harbor database

 

After starting the replication the Harbor database pod is shutting down with Out Of Memory:

 

kubectl get events -n svc-harbor-domain-c####

Events:
  Type     Reason                        Age   From                     Message
  ----     ------                        ----  ----                     -------
  Normal   Scheduled                     24m   default-scheduler        Successfully assigned svc-harbor-domain-c############/harbor-database-0 to ############
  Normal   Image                         24m   image-controller         Image harbor-############ bound successfully
  Normal   SuccessfulRealizeNSXResource  24m   nsx-container-ncp        Successfully realized NSX resource for Pod
  Normal   SuccessfulAttachVolume        24m   attachdetach-controller  AttachVolume.Attach succeeded for volume "pvc-############"
  Normal   Pulling                       24m   kubelet                  Waiting for Image svc-harbor-domain-c############/harbor-############
  Normal   Pulled                        24m   kubelet                  Image svc-harbor-domain-c############/harbor-############ is ready
  Normal   SuccessfulMountVolume         24m   kubelet                  Successfully mounted volume database-data
  Normal   Created                       24m   kubelet                  Created container data-permissions-ensurer
  Normal   Started                       24m   kubelet                  Started container data-permissions-ensurer
  Normal   Created                       24m   kubelet                  Created container database
  Normal   Started                       24m   kubelet                  Started container database
  Normal   SuccessfulMountVolume         24m   kubelet                  Successfully mounted volume shm-volume
  Warning  InsufficientFreeMemory        16s   kubelet                  Detected OOM situation, shutting down Pod

 

Environment

vSphere 8.0, Harbor  2.13.1

Cause

Default memory resources for database in Harbor version 2.1.3.1 are not sufficient for replication 

Resolution

Upgrade to Harbor 2.14.2 in order to be able to configure resources for the Harbor pods 

Note: Configuring resources in Harbor 2.13.1 is not supported

 

The default memory resources for the database in Harbor 2.14.2  are 1Gi (which should be enough memory for replication)

 

Harbor 2.14.2 Package Reference

harbor-data-values-v2.14.2

resources:
#! The settings for LimitRange of Harbor's namespace
  namespaceLimitRange:
    defaultCPURequest: 300m
    defaultCPULimit: 1000m
    defaultMemoryRequest: 256Mi
    defaultMemoryLimit: 2Gi
#! The resource settings for each of the containers of Harbor
  containers:
    - name: core
      cpuRequest: 500m
      cpuLimit: 1000m
      memoryRequest: 512Mi
      memoryLimit: 2Gi
    - name: database     
      cpuRequest: 1000m
      cpuLimit: 2000m
      memoryRequest: 1Gi
      memoryLimit: 2Gi
    - name: jobservice
      cpuRequest: 500m
      cpuLimit: 1000m
      memoryRequest: 512Mi
      memoryLimit: 2Gi
    - name: portal
      cpuRequest: 100m
      cpuLimit: 300m
      memoryRequest: 128Mi
      memoryLimit: 256Mi
    - name: redis
      cpuRequest: 200m
      cpuLimit: 500m
      memoryRequest: 256Mi
      memoryLimit: 1Gi
    - name: registry
      cpuRequest: 500m
      cpuLimit: 1000m
      memoryRequest: 512Mi
      memoryLimit: 2Gi
    - name: registryctl
      cpuRequest: 200m
      cpuLimit: 500m
      memoryRequest: 256Mi
      memoryLimit: 512Mi
    - name: trivy
      cpuRequest: 500m
      cpuLimit: 2000m
      memoryRequest: 1Gi
      memoryLimit: 4Gi
    - name: exporter
      cpuRequest: 100m
      cpuLimit: 300m
      memoryRequest: 128Mi
      memoryLimit: 256Mi