Multiple causes exist for this symptom. Full evaluation requires in depth analysis of proper vlan tagging, trunk configurations, and network connectivity between the ESXi hosts and the physical infrastructure.
See the following KB articles for various troubleshooting / investigation paths:
NSX-MGR-CONNECT