When accessing a particular ingress URL in TKGI, you see a 502 Bad Gateway error, even though the related ingress, pod, and service are all in running state. Meanwhile, other ingress URLs in the same namespace respond successfully with a 200 OK status.
An example 502 Bad Gateway error:
# curl -vvv -i https://INGRESS-URL -k
...
< HTTP/1.1 502 Bad Gateway
HTTP/1.1 502 Bad Gateway
< Server: NSX LB
Server: NSX LB
< Date: Wed, 14 May 20## 02:25:59 GMT
Date: Wed, 14 May 20## 02:25:59 GMT
< Content-Type: text/html
Content-Type: text/html
< Content-Length: 151
Content-Length: 151
< Connection: keep-alive
Connection: keep-alive
<
<html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>NSX LB</center>
</body>
</html>
* Connection #0 to host INGRESS-URL left intact
When testing the corresponding pods using kubectl port-forwarding the attempt failed with a connection refused error, followed by lost connection to pod.
Use with pod targetPort 1081 as an example:
$ kubectl port-forward -n NAMESPACE POD-#### 8088:1081
Forwarding from 127.0.0.1:8088 -> 1081
Handling connection for 8088E0519 06:56:56.323411 1556492 portforward.go:413] an error occurred forwarding 8088 -> 1081: error forwarding port 1081 to pod 3828########################################################ac94, uid : failed to execute portforward in network namespace "/var/run/netns/cni-########-####-####-####-########851b": failed to connect to localhost:1081 inside namespace "3828########################################################ac94", IPv4: dial tcp4 127.0.0.1:1081: connect: connection refused IPv6 dial tcp6 [::1]:1081: socket: address family not supported by protocol
error: lost connection to pod
Tanzu Kubernetes Grid Integrated Edition
Port forwarding in k8s is typically used to access a pod directly, bypassing the ingress layers. The port forwarding error suggests that the pod might not be accepting connections or there could be an issue with networking.
Ensure the problematic pod port is open and accepting connections by properly configuring the application to listen on the designated port and checking firewall settings and network policies to prevent any restrictions.