Error: couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused
search cancel

Error: couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused

book

Article ID: 387272

calendar_today

Updated On: 06-27-2025

Products

Tanzu Kubernetes Grid

Issue/Introduction

kubectl command

couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp 127.0.0.1:8080: connect: connection refused

s results in error:

The Kubernetes API server container is in an "exited" state, and attempts to connect to the KubeVIP are unsuccessful by running:
  1. SSH into the host
    ssh capv@<managment node ip>
  2. Find the etcdctl executable:
    find /var -name etcdctl
  3. Create an alias for etcdctl
    alias etcdctl=<path from output of find command in step 2>
  4. Get members of the ETCD cluster
    etcdctl member list
  5. The output of "etcdctl member list" does not show any members and "crictl ps -a" command shows kube-apiserver container in exited status
    [ ~ ]# crictl ps -a
    CONTAINER       IMAGE       CREATED      STATE            NAME            ATTEMPT             POD ID              POD
    #####           #####       <time>       Exited           kube-apiserver  ####                ####                                <pod name>



Environment

2.x

Cause

A mismatch was identified between the IPs in the etcd.yaml file and the IPs assigned to the control plane VMs. The IPs originally assigned to the control plane nodes were changed, which caused a loss of the etcd quorum.

 


The cloud-init output log (cat /var/log/cloud-init-output.log) to confirm the original IPs assigned to the control plane manager vm differ from current ips shown in vcenter server > inventory vm > summary > ip address.

output of kubectl get nodes -o wide showed same result and the etcd.yaml file located at "/etc/kubernetes/manifests/etcd.yaml" under the "--listen-peer-urls" field reflected these same mismatched IPs.

The mismatch between the assigned IPs and the original IPs of the VMs caused the etcd quorum to be lost.

The ip change can happen when DHCP leases are not statically assigned to control plane vms during provisioning.

 

Resolution

1. Reassign the original IPs to the control plane VMs

2. Update the DHCP server to reflect the correct IP and lease time assignments

or

3. Manually update on the vm (if a photon deployed control plane vm):

a. vi command on  /etc/systemd/network/<network file> and press i to update the file with the following parameters

        i. DHCP=no

        ii. address=<noted address from etcd.yaml>

        iii. gateway=<gateway address>

        iv. confirm settings and saved using :wq!

4. Restart network using:
    systemctl restart networkd-systemd

5. Confirm Ip address is reflected correctly on all management control plane vms

6. Confirm etcd is restarted using command from (above step 4 on issue): etcdctl member list

7. Confirm kube-apiserver as well using crictl ps

Once etcd and kube apiserver are started other services should also begin to startup.

 

Additional Information

For more information on manual ip update see Setting a static ip address

Information regarding kubernetes etcd see Configure-upgrade-etcd