Pod to pod communication isn't working when setting deployment's hostNetwork setting to true.

Products

• VMware vSphere Kubernetes Service

Issue/Introduction

You have multiple Kubernetes deployments where you set hostNetwork: true and the pods are unable to communicate with each other.

Environment

vSphere Kubernetes Service

Cause

Setting hostNetwork: true tells the pod network traffic to bypass the Container Network Interface (CNI) (Antrea / Calico) and to use the node's operating system networking stack instead. This traffic is filtered based off of the operating system's iptables configuration, which is configured to only allow certain traffic. If your pods are communicating over a protocol/port that isn't allowed, the traffic will be dropped.

Resolution

On the Kubernetes worker nodes, the file /etc/systemd/scripts/ip4save that contains all the nodes's iptable firewall rules. One of those rules is:

-A INPUT -p tcp -m multiport --dports 30000:32767 -j ACCEPT

This says that any TCP port from 30000 to 32767 is allowed on the node. When you have a requirement to set hotsNetwork to true, you have to use ports within this range.