PKS cluster creation in NSX-T environment fails with createIpBlockSubnetConflict

Products

• VMware Tanzu Kubernetes Grid Integrated Edition

Issue/Introduction

Symptoms:
Cluster creation fails with the following error:

pks create-cluster oom --external-hostname outofmemory --plan small -n 1

Error: Error creating NSX-T cluster network `pks-6f3af6da-a55c-4e1e-9396-310d97fe9302`: [POST /pools/ip-subnets][409] createIpBlockSubnetConflict 

Environment

Cause

Starting PKS 1.1 network objects, T-1 router, logical switches etc., that host Kubernetes vms are created automatically when pks create-cluster is executed. PKS allocates IP addresses for the nodes when they are created in Kubernetes. The node networks are created on a separate IP address space from the pod networks. These block used for these IP addresses are declared under Networking on PKS Tile. Nodes IP Block is defined under Node IP block. Here is the path to reach: Go to NSX-T Manager, select DDI, select IPAM, select Node IP block. Each cluster uses a subnet size of /24. If you want to create multiple clusters Node IP block subnet size should be a multiple of /24. For example - if the Node IP block has a CIDR 172.26.0.0/16 a maximum of 256 clusters can be created.

createIpBlockSubnetConflict can occur in the following scenarios:

• If Node IP block has a CIDR range of /24 and a second cluster creation is attempted
• If a previous cluster creation attempt has failed and Node IP block has a CIDR range of /24

Resolution

Following are the solutions for the two scenarios discussed above:

• If Node IP block has a CIDR range of /24 and a second cluster creation is attempted
  • ​Solution: Increase the CIDR range under Node IP Block. Go to NSX-T Manager, select DDI, select IPAM, select Node IP block.
• If a previous cluster creation attempt has failed and Node IP block has a CIDR range of /24
  • ​Solution: There is a subnet objects created under Subnets. Here is how to reach there: Go to NSX-T Manager, select DDI, select IPAM, select Node IP block, click Subnets. Removed the object for the failed cluster.