vmware-system-vmop-controller-manager (VM operator) pods are restarting and remain stuck in a CrashLoopBackOff state.
Describing an affected vmware-system-vmop-controller-manager pod returns a container is failing due to OOMKilled:
kubectl describe pod -n <namespace> <pod name>finishedAt: "YYYY-MM-DDTHH:MM:SSZ"
reason: OOMKilled startedAt: "YYYY-MM-DDTHH:MM:SSZ"vSphere 8.x
vSphere Supervisor
The affected VM operator pods default memory limits are unable to keep up with the large amount of resources needed by a large vSphere Supervisor environment.
Workaround:
Increase the memory limit of the VM Operator pod in 50Mi increments until the pods start and remain in a Running status.
The following steps detail how to increase the limit to 600Mi (MB) or higher as needed:
1. SSH into the vCenter appliance:
2. Follow KB article 90194 to connect into the Supervisor Control Plane VM as root via SSH.
3. Describe the vmware-system-vmop-controller-manager deployment and determine the existing memory configuration:
Containers: manager: Image: localhost:5000/vmware/vmop:0.1.0-bdffdb8-1.8.6-########-######## Ports: 9878/TCP, 9445/TCP Host Ports: 0/TCP, 0/TCP Command: /manager Args: --metrics-addr=127.0.0.1:8083 Limits: cpu: 750m memory: 550Mi
4. Scale down the VM Operator deployment so no pods are running:
kubectl -n vmware-system-vmop scale deployment vmware-system-vmop-controller-manager --replicas 0
5. Increase the memory limit by 50Mi for the VM Operator pod using the following command (this example is increasing the original 550Mi configuration to 600Mi):
kubectl -n vmware-system-vmop patch deployment vmware-system-vmop-controller-manager -p '{"spec":{"template":{"spec":{"containers":[{"name":"manager","resources":{"limits":{"memory":"600Mi"}}}]}}}}'
6. Scale the replicas back to their original value:
kubectl -n vmware-system-vmop scale deployment vmware-system-vmop-controller-manager --replicas 2
If OOMKilled errors are still encountered, increase the memory limit in increments of 50Mi until the VMOP pods run normally.