HCX MON Asymmetrical Routing Causes SentinelOne agent connectivity Failures for migrated VMs

Products

• VMware HCX

Issue/Introduction

• Virtual machines migrated to VMware Cloud on AWS (VMC) on an HCX Layer 2 Extension (L2E) experience connectivity issues with a SentinelOne agent.
• Virtual machines with Mobility Optimized Networking (MON) enabled and the router set to the cloud fail to communicate (towards Internet). 
• Virtual machines without MON enabled, or those using the on-premises router, operate as expected.

Environment

VMware HCX 4.11.3

VMware Cloud on AWS (VMC)

Cause

Asymmetrical routing occurs when MON is enabled. Outbound traffic successfully egresses via the cloud NSX gateways (T1/T0/Edge), but the external core network routes the return traffic to the on-premises gateway instead of the VMC environment.

Resolution

Implement one of the following methods to resolve the routing asymmetry:

Option 1: Route via On-Premises Route the affected traffic through the HCX L2E tunnels so that virtual machines in the cloud egress via the on-premises gateway.

Option 2: Correct Return Routing on External Cloud Infrastructure Modify the routing tables on the external cloud core switch/router to ensure return ingress traffic destined for the MON-enabled segments is directed to the VMC NSX environment (T0/Edge) instead of the on-premises gateway.

Option 3: Perform a Gateway Cutover

1. Pre-Cutover Validation:

   • Ensure all virtual machines requiring the gateway are migrated to the target site.

   • Validate workloads depending on source site DHCP, DNS, or NTP services are reconfigured for the target site infrastructure.

   • Ensure dynamic routing (BGP/OSPF) is correctly configured on the target site Edge Gateway/Tier-1 Router.

2. Execute the Gateway Cutover:

   • Log in to the source HCX Manager UI.

   • Navigate to Services > Network Extension.

   • Locate the specific network undergoing the cutover and expand the entry.

   • Select the checkbox for Connect cloud network to cloud edge gateway after unextending.

   • Click Unextend.

3. Post-Cutover Verification:

   • Log in to the target site NSX Manager.

   • Navigate to Networking > Segments.

   • Locate the cutover segment and verify the Gateway Connectivity state is Enabled.

   • Validate North-South and East-West routing for the workloads on the segment.

Additional Information

Mobility Optimized Networking (MON) Troubleshooting Guide

Removing a Network Extension