TKGI / PKS login command fails with error: net/http Client timed out while awaiting headers
search cancel

TKGI / PKS login command fails with error: net/http Client timed out while awaiting headers


Article ID: 298722


Updated On:


VMware Tanzu Kubernetes Grid Integrated Edition


Executing the TKGI (formerly PKS) login command fails with the following error:

Note: Make sure that you are seeing an IP address in the error. If an IP address is not in the error then the cause of timeout will be different and this article might not help to address that particular issue.
$ tkgi login -k -a <IP-Address> -u <username>
Password: *******

Error: Post net/http: request canceled (Client.Timeout exceeded while awaiting headers)

Upon checking the logs of the TKGI API VM, specifically the pks-api monit jobs logs, you see the following error and its stack trace (filename: /var/vcap/sys/log/pks-api/pks-api.log):
2021-05-12 18:35:54.447 ERROR 27073 --- [nio-9021-exec-4] o.a.c.c.C.[Tomcat].[localhost]           : Exception Processing ErrorPage[errorCode
=0, location=/error] org.springframework.web.client.ResourceAccessException: I/O error on POST request for "": Hostname not verified:
    certificate: sha256/redacted
    DN: <redacted>... ...<redacted>
    subjectAltNames: []; nested exception is Hostname not verified:
    certificate: sha256/<redacted>
    DN: <redacted>... ...<redacted>
    subjectAltNames: []
        at org.springframework.web.client.RestTemplate.doExecute( ~[spring-web-5.1.18.RELEASE.jar!/:5.1.18.RELEASE]
        at org.springframework.web.client.RestTemplate.execute( ~[spring-web-5.1.18.RELEASE.jar!/:5.1.18.RELEASE]
        at ~[spring-web-5.1.18.RELEASE.jar!/:5.1.18.RELEASE]
        at ~[spring-security-oauth2-2.3.6.RELEASE.jar!/:na]

The above error suggests that the certificates for the TKGI API is signed for hostname and does not have any alternative names (subjectAltNames: []). As you have specified an IP address while logging in to the API server, the hostname could not be verified in the certificate. This leads to login attempts being considered a failure and ultimately, a timeout with the client timed out and awaiting headers.


Product Version: 1.9
OS: Linux


To fix the issue, make sure to specify a hostname as opposed to an IP address at the location Ops Manager UI -> TKGI tile -> TKGI API -> API Hostname (FQDN). Upon saving the change, run an Apply Changes on the BOSH Director tile + TKGI tile (without the upgrade all instances errand) so the changes can be propagated to the API server VM. 

The example shown in the Issue section previously was fixed by changing the API Hostname (FQDN) in the tile from IP to (the certificate was also generated for hostname