Service of type load balancer creation is failing in TKGI with ncp error "NCP00022"
Article ID: 375651
Updated On:
Products
Issue/Introduction
When creating service of type load balancer will fail with below ncp error log in the master node.
NSX 1346092 - [nsx@6876 comp="nsx-container-ncp" subcomp="ncp" level="WARNING"] vmware_nsxlib.v3.client The HTTP request returned error code 40
0, whereas 201/200 response codes were expected. Response body {'httpStatus': 'BAD_REQUEST', 'error_code': 10000, 'module_name': 'ROUTING', 'error_message': 'Found errors in the request. Please refer to the related errors for details.', 'related_errors': [{'httpStatus': 'BAD_REQUEST', 'error_code': 10087, 'module_name': 'ROUTING', 'error_message': '[Routing] Insufficient resources to do auto allocation in edge cluster ########-####-4gf3-b77d-c876bec5t3678 of pool type LB_ALLOCATION_POOL because of below reason(s).'}, {'httpStatus': 'BAD_REQUEST', 'error_code': 10149, 'module_name': 'ROUTING', 'error_message': "[Routing] Node(s) [EdgeTransportNode/########-####-478f-78cc-ae6d779ac551, EdgeTransportNode/########-####-423c-94ea-4f8f6t867d45] don't have enough capacity to allocate."}]}
NSX 1346092 - [nsx@6876 comp="nsx-container-ncp" subcomp="ncp" level="ERROR" security="True" errorCode="NCP00022"]
Running the command "kubectl get svc example -o yaml" will give the bellow error o/p
{"apiVersion":"v1","kind":"Service","metadata":{"annotations":{},"name":"etaxcourt-service","namespace":"eksternalservice"},"spec":{"ports": [{"name":"http","port":8080,"protocol":"TCP","targetPort":8080}],"selector":{"app":"etaxcourt-service"},"type":"LoadBalancer"}} ncp/error.loadbalancer: SERVICE_LOADBALANCER_UNREALIZED
Environment
TKGI/NSX-T
Cause
This is due to insufficinet LB resource in the edge cluster.
Validate the edge nodes utilization in the edge cluster using the below command.
curl -k -X GET -u 'admin:password' https://<NSX_MANAGER_FQDN>/api/v1/loadbalancer/usage-per-node/########-####-478f-78cc-ae6d779ac551
{
"form_factor" : "LARGE_VIRTUAL_MACHINE",
"edge_cluster_id" : "########-####-4ba3-b67d-c567bec3d369",
"current_credit_number" : 40,
"remaining_credit_number" : 0,
"usage_percentage" : 100.0,
"severity" : "RED",
"current_pool_members" : 500,
"current_virtual_servers" : 560,
"current_pools" : 300,
"current_small_load_balancer_services" : 0,
"current_medium_load_balancer_services" : 4,
"current_large_load_balancer_services" : 0,
"current_xlarge_load_balancer_services" : 0,
"remaining_small_load_balancer_services" : 0,
"remaining_medium_load_balancer_services" : 0,
"remaining_large_load_balancer_services" : 0,
"remaining_xlarge_load_balancer_services" : 0,
"remaining_pool_members" : 4538,
"type" : "LbEdgeNodeUsage",
"node_id" : "########-####-478f-78cc-ae6d779ac551"
}
-> Above o/p indicates edge node is large form factor and 4 medium loadbalancer is created.
-> In a large edge node only 4 medium seized lb can be created. To verify the max limits please check the below link.
https://configmax.esp.vmware.com/guest?vmwareproduct=&release=&categories=20-0
-> In this case max limit has been reached on all the edge nodes in the edge cluster
Resolution
Adding the edge node to the edge cluster.
Note: An Edge Cluster can have a maximum of 10 Edge Nodes.
Additional Information
When a new Kubernetes cluster is provisioned using the TKGI API, NSX creates a dedicated load balancer for that new cluster. By default, the size of the load balancer is set to Small.
With network profiles, you can change the size of the load balancer deployed by NSX at the time of cluster creation.
If the size of the load balancer is medium while creating the cluster. Then the service of type lb will also created with medium sized load balancer. This can be validated in the ncp.ini config in the master nodes.
/var/vcap/jobs/ncp/config/ncp.ini
service_size = MEDIUM
Feedback
