Tanzu Kubernetes Grid management cluster creation fails with reason:'BastionHostFailed', message:'6 of 8 completed'
searchcancel
Tanzu Kubernetes Grid management cluster creation fails with reason:'BastionHostFailed', message:'6 of 8 completed'
book
Article ID: 319315
calendar_today
Updated On: 06-18-2024
Products
Tanzu Kubernetes Grid
Issue/Introduction
Symptoms:
You are attempting to create a TKG management cluster in AWS and it fails during the creation of bastion host
You observe the below error returned by the cli
unable to set up management cluster: unable to wait for cluster and get the cluster kubeconfig: error waiting for cluster to be provisioned (this may take a few minutes): cluster creation failed, reason:'BastionHostFailed', message:'6 of 8 completed'
Environment
VMware Tanzu Kubernetes Grid 1.x
Cause
To find the root cause of this problem, you should look at the capa-controller-manager pod logs in the capa-system namespace:
If you see the below error in your logs, the error occurs because the Bastion host AMI lookup fails in the region where you are creating your cluster.
E1221 23:21:57.608603 1 controller.go:257] controller-runtime/controller "msg"="Reconciler error" "error"="failed to reconcile bastion host for AWSCluster tkg-system/oom: failed to run instance: InvalidAMIID.Malformed: Invalid id: \"unknown region\" (expecting \"ami-...\")\n\tstatus code: 400, request id: #####" "controller"="awscluster" "name"="oom" "namespace"="tkg-system"
You will observe this error if you are deploying a cluster in a region where bastion host AMI is not available. us-gov-west is an example of such a region.
Resolution
You must switch to one of the regions where bastion host AMI is available. You can get an exhaustive list of regions where AMI lookup is done at cluster-api-provider-aws/pkg/cloud/services/ec2/ami.go Please refer to function defaultBastionAMILookup for the version of cluster-api-provider-aws shipped with your TKG version.
An alternate option is to disable bastion host creation. This can be done from the TKG installer UI and also by setting BASTION_HOST_ENABLED: "false" in the TKG configuration.