Pointed a fluxcd GitRepository at a repo larger than 1G and the fluxcd-source-controller crashed with OOMKilled 137.

Products

• VMware Tanzu Application Platform

Issue/Introduction

If you point a fluxcd GitRepository at a repo larger than 1G, the fluxcd-source-controller will crash with OOMKilled 137.

From the output of kubectl describe pod,

• Last State: Terminated
• Reason: OOMKilled
• Exit Code: 137
• Limits.memory: 1Gi

Resolution

The memory and cpu resources can be updated in the tap values, for example:

fluxcd_source_controller:
  resources:
    limits_memory: 3Gi

 

You may find that increasing the concurrency of the fluxcd controller is also helpful in increasing its throughput. That is not configurable through the tap values but can be updated with an overlay:

apiVersion: v1
kind: Secret
metadata:
  name: fluxcd-source-overlay
  namespace: tap-install
stringData:
  fluxcd-source-overlay.yaml: |
    #@ load("@ytt:overlay", "overlay")
    #@overlay/match by=overlay.subset({"kind":"Deployment", "metadata":{"name":"fluxcd-source-controller", "namespace": "flux-system"}}), expects="1+"
    ---
    spec:
      template:
        spec:
          containers:
          #@overlay/match by="name"
          - name: manager
            resources:
              limits:
                memory: "4Gi"
              requests:
                cpu: "2"
                memory: "2Gi"
            args:
            - --concurrent=10
            - --requeue-dependency=5s