Configuring VMware vSphere Security Policies for Windows Network Load Balancing (NLB) While Minimizing Security Exposure
search cancel

Configuring VMware vSphere Security Policies for Windows Network Load Balancing (NLB) While Minimizing Security Exposure

book

Article ID: 414968

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

When deploying Microsoft Windows Network Load Balancing (NLB) on VMware vSphere, specific virtual switch security policies (Promiscuous mode, Forged Transmits, and MAC Address Changes) must often be set to Accept on the port group to which the NLB virtual machines (VMs) are connected. This requirement is crucial for NLB to function correctly in both unicast and multicast modes. However, IT administrators frequently raise concerns about the security implications of enabling these settings, particularly the potential for packet sniffing, MAC spoofing, and network instability. This article clarifies the configuration, associated risks, and best practices to minimize security exposure.

Environment

VMware ESX

Cause

Windows NLB requires specific virtual switch security policies to be enabled due to its operational mechanisms for handling network traffic:

  • Promiscuous mode = Accept: Required primarily for NLB in multicast mode. In this mode, all NLB cluster members share the same IP and MAC address. The virtual switch, by default, will not forward frames destined for the NLB cluster MAC address to all members. Enabling promiscuous mode allows all NLB VMs on the port group to receive all traffic, including that destined for the shared NLB MAC.
  • Forged Transmits = Accept: Required for both unicast and multicast modes. NLB VMs often send out packets with a source MAC address that is the shared NLB cluster MAC, which differs from the individual VM's assigned MAC address. By default, vSphere drops packets where the source MAC address doesn't match the VM's assigned vNIC MAC. Forged Transmits = Accept allows the VM to send these packets.
  • MAC Address Changes = Accept: Required primarily for NLB in unicast mode. In this mode, all NLB cluster members share the same IP, but traffic for the cluster IP is often handled using a virtual MAC address configured within the guest OS. This virtual MAC address differs from the MAC address assigned by vSphere to the virtual NIC. MAC Address Changes = Accept allows the guest OS to use its self-configured MAC address.

Resolution

To enable Windows NLB functionality while mitigating security risks, Broadcom (VMware) recommends the following strategy and best practices:

1. Recommended Configuration (VMware-Recommended and Significantly Safer):
         (https://knowledge.broadcom.com/external/article/344421/microsoft-nlb-not-working-properly-in-un.html)

  • Create a Dedicated Distributed Port Group: Create a new, isolated distributed port group exclusively for your Windows NLB VMs. Do not attach any other production VMs to this port group.
  • Apply Policies Sparingly: Only apply the necessary "Accept" settings to this dedicated NLB port group:
    * For NLB Unicast: Set MAC Address Changes = Accept and Forged Transmits = Accept.
    * For NLB Multicast: Set Promiscuous mode = Accept and Forged Transmits = Accept.
  • Keep Others Default: Ensure all other production port groups maintain their default (Reject) security policy settings. This prevents these settings from affecting unrelated VMs.

2. Understanding and Mitigating Security Risks:

Additional Information

Powered by Wolken Software