ako-controller-manager pod keeps crashing in vSphere with Tanzu

Products

• vSphere with Tanzu
• VMware NSX
• VMware NSX Advanced Load Balancer

Issue/Introduction

• alb-endpoint was unregistered while ncp was restarted
• avi-secret is missing in vSphere with Tanzu
• ako-controller-manager pod logs showing using nsx-lb as LBProvider

  [ncp MainThread I] nsx_ujo.ncp.nsx.nsx_config Validated LoadBalancerProvider, its value is nsx-lb

Environment

• vSphere 8 using Supervisor networking with NSX and NSX Advanced Load Balancer environment

Cause

NCP logic to choose NSX-ALB:

1. use_avi_lb is true in ConfigMap
2. ALB endpoint is registered on NSX
3. There is no LBService created on NSX for this cluster

When alb-endpoint is unregistered and the ncp pod is restarted, ncp will create LBServices in NSX.
After alb-endpoint is registered again in NSX, ncp will find existing LBServices in NSX and choose NSX as LBResources, therefore avi-secret is not being created, causing ako-controller-manager pod to fail to start.

Resolution

If you believe you have encountered this issue, please open a support case with Broadcom Support and refer to this KB article.

For more information, see Creating and managing Broadcom support cases.