Microsoft Windows Server Failover Clustering (WSFC) with shared disks on VMware vSphere 6.x: Guidelines for supported configurations
search cancel

Microsoft Windows Server Failover Clustering (WSFC) with shared disks on VMware vSphere 6.x: Guidelines for supported configurations

book

Article ID: 315481

calendar_today

Updated On:

Products

VMware vSphere ESXi

Issue/Introduction

NOTE: The content of the Knowledge Base Article has been changed significantly. This new content completely replaces the old revision of the KB.

VMware provides customers flexibility and choice in architecting high-availability solutions using Windows Server as a guest operating systems (OS). VMware vSphere supports Windows Server Failover Cluster (WSFC) with shared disk by transparently passing to the underlying storage SCSI-3 Persistent Reservations (SCSI3-PRs) SCSI commands, required for WSFC node (VM participating in a WSFC, further references as VM node) to arbitrate an access to a shared disk. It is a general best practice to ensure that each node participating in WSFC has the same configuration.

The information in this article is applicable to configurations when VMs hosting nodes of a WSFC (VM nodes) are located on different ESXi hosts – known as “Cluster-across-box (CAB)” in VMware official documentation. CAB provides high availability (HA) both from the in-guest and vSphere environment perspective. We do not recommend a configuration where all VM nodes are placed on a single ESXi host (so called “cluster-in-a-box”, or CIB). The CIB solution should not be used for any production implementations – if a single ESXi host will fail, all cluster nodes will be powered off and, as the result, an application will experience downtime.

This KB assumes that all components underneath the WSFC-based VMs are architected to provide the proper availability and conform to Microsoft’s supportability as it relates to the in-guest configuration.

NOTE: A single WSFC consisting of both physical nodes and virtual machine is supported by both Microsoft and VMware. For more information, select your version from https://kb.vmware.com/s/article/1004617 and review the "Cluster Physical and Virtual Machines" section in the Setup for Failover Clustering and Microsoft Cluster Service Guide.


Environment

VMware vSphere ESXi 6.7
VMware vSphere ESXi 6.0
VMware vSphere ESXi 6.5

Resolution

vSphere Version Support for WSFC

Table 1 shows supported versions of Windows OS and VMware vSphere, being qualified by VMware. VMware doesn't impose any limitation nor require a certification for applications using WSFC on a supported Windows platform. Therefor any applications running on a supported combination of vSphere and Windows OS is supported with no additional considerations.

NOTE: Other WSFC-based solutions not accessing shared storage (SQL Server Always On Availability Groups (AGs) or Exchange Database Availability Group (DAG)) require no special storage configurations on the vSphere side (VMFS or NFS). This KB should not be used for such configurations.

Table 1. Versions of Windows Server Supported by vSphere for a WSFC
 

Windows Server Version1

Minimum vSphere version

Maximum Number of WSFC Nodes with Shared Storage Supported by ESXi

2019

vSphere 6.5 Update 3, vSphere 6.7 Update 3

5

2016

vSphere 6.7, vSphere 6.5, vSphere 6.0

5

2012 / 2012 R22

vSphere 6.7, vSphere 6.5, vSphere 6.0

5

2008 R23

vSphere 6.7, vSphere 6.5, vSphere 6.0

5

 
  1. SQL Server 2016 and 2017 Failover Cluster Instances (FCI) were used to validate the WSFC functionality on vSphere and Windows Server versions listed in this table.
  2. If the cluster validation wizard completes with the warning: ”Validate Storage Spaces Persistent Reservation,” you can safely ignore the warning. This check applies to the Microsoft Storage Spaces feature, which does not apply to VMware vSphere.
  3. Windows Server 2008 / 2008 R2 release have reached the end of extended support (no regular security updates). While still qualified on the vSphere platform, consider the vendor supportability while hosting a WSFC on VMware vSphere.


VMware vSphere features support for WSFC Configurations with shared disks

Following VMware vSphere features are supported for WSFC:
  • VMware HA. DRS Affinity rules required. When creating a DRS Affinity rule, select Separate Virtual Machines. For more details consult the documentation
  • VMware DRS. See the Support requirements for vMotion of a VM hosting a node of WSFC below.
  • Offline (cold) Storage vMotion of a VM, hosting a node of WSFC with shared storage is supported when the target destination is vSAN 6.7 Update 3 or VMware Cloud on AWS.
Following VMware vSphere features are NOT supported for WSFC:
  • Live Storage vMotion support
  • Fault Tolerance
  • N-Port ID Virtualization (NPIV)
  • Mixed versions of ESXi hosts in a vSphere cluster


Support requirements for vMotion of a VM hosting a node of WSFC

Live vMotion (both user- or DRS-initiated) of VM nodes is supported in vSphere 6.0 or later with the following requirements:
  • The virtual hardware version must be version 11 (vSphere 6.0) and later.
  • The value of the SameSubnetThreshold Parameter of Windows cluster health monitoring must be modified to allow 10 missed heartbeats at minimum. This is the default in Windows Server 2016 . This recommendation applies to all applications using WSFC, including shared and non-shared disks.
  • Must configure cluster nodes with a DRS Affinity rule to prevent hosting more than one node of a WSFC cluster on a single ESXi host. This means that you must have N+1 ESXi hosts, where N is the number of WSFC nodes.
  • The vMotion network must be using a physical network wire with a transmission speed 10GE (Ten Gigabit Ethernet) and more. vMotion over a 1GE (One Gigabit Ethernet) is not supported.
  • Shared disks resources must be accessible by the destination ESXi host.
  • Supported version of Windows Server used (see Table 1 for the list of supported versions of Windows Server).


Storage Configuration

The vSphere options for presenting shared storage to a VM hosting a node of WSFC are shown in Table 2.

Table 2. Supported storage configuration options
 

vSphere version

Shared disk options

SCSI bus sharing

vSCSI Controller type

Storage Protocol / Technology

vSphere 6.0 and 6.5

RDM physical mode

physical

VMware Paravirtual (PVSCSI), LSI Logic SAS

FC, FCoE, iSCSI

vSphere 6.7

VMware vSphere® Virtual Volumes (vVols), RDM physical mode,

physical

VMware Paravirtual (PVSCSI), LSI Logic SAS

FC, FCoE, iSCSI

VMware Cloud on AWS

Clustered VMDK

physical

VMware Paravirtual (PVSCSI), LSI Logic SAS

N/A / vSAN

vSAN (vSphere 6.7 U3)

Clustered VMDK

physical

VMware Paravirtual (PVSCSI), LSI Logic SAS

N/A / vSAN


Multipathing configuration : Path Selection Policy (PSP)

  • RDM physical mode: Round Robin PSP is fully supported. Fixed and MRU PSPs can be used as well, but Round Robbin PSP might provide better performance utilizing all available paths to the storage array.
  • vVols: Fixed and MRU PSPs starting with vSphere 6.7 GA, Round Robbin PSP support introduced in vSphere 6.7 Update 2.
NOTE: While choosing a PSP to use, consult your storage array vendor for the recommended/supported PSP. For more information, see the Storage/SAN Compatibility Guide .


Perennially reservations

VMware recommends implementing perennial reservation for all ESXi hosts hosting VM node with clustered disk resources. Check this Knowledge Base Article for more details.


Virtual SCSI Controllers

1. Mixing non-shared and shared disks

Mixing non-shared and shared disks on a single virtual SCSI adapter is not supported. For example, if the system disk (drive C:) is attached to SCSI0:0, the first clustered disk would be attached to SCSI1:0. A VM node of a WSFC has the same virtual SCSI controller maximum as an ordinary VM - up to four (4) virtual SCSI Controllers.


2. Modify advanced settings for a virtual SCSI controller hosting the boot device.

Add the following advanced settings to the VMs node:

scsiX.returnNoConnectDuringAPD = "TRUE"
scsiX.returnBusyOnNoConnectStatus = "FALSE"

Where X is the boot device SCSI bus controller ID number. By default, X is set to 0.

3. Virtual discs SCSI IDs should be consistent between all VMs hosting nodes of the same WSFC.
4. Use VMware Paravirtual virtual SCSI controller for best performance.
 
For the best performance consider distributing disks evenly between as much SCSI controllers as possible and use VMware Paravirtual (PVSCSI) controller (provides better performance with lower CPU usage and is a preferred way to attach clustered disk resources).
 

Virtual Disk Configuration

RDMs used as clustered disk resources must be added using the physical compatibility mode.


Storage protocols

  • FC, FCoE and Native iSCSI are fully supported.
  • NFS is not a supported storage protocol to access a clustered disk resource for WSFC. NFS backend VMDKs can be used as non-shared disks (system disk, backup, etc.) without limitations.
  • vSAN supports natively clustered VMDK starting with the version 6.7 Update 3 and provides an iSCSI target service starting with vSAN 6.7 GA. Check this Knowledge Base Article for more details.
  • Virtual Volumes – supported starting with vSphere 6.7 GA. Check with your storage vendor if the implementation of vVol includes support for WSFC on VMware vSphere.


In-guest shared storage configuration options

Maintaining in guest options for storage (such as iSCSI or SMB shares) is up to those implementing the solution and is not visible to ESXi.

VMware fully supports a configuration of WSFC using in-guest iSCSI initiators or in-guest SMB (Server Message Block) protocol, provided that all other configuration meets the documented and supported WSFC configuration. Using this configuration in VMware virtual machines is similar to using it in physical environments.

NOTE: vMotion has not been tested by VMware with any in-guest shared storage configurations.


VM Limitations when hosting a WSFC with shared disk on vSphere

Hot changes to virtual machine hardware might disrupts the heartbeat between the WSFC nodes. The following activities are not supported and might cause WSFC node failover:

  • Hot adding memory
  • Hot adding CPU
  • Using snapshots.
  • Increasing the size of a shared disk
  • Pausing and/or resuming the virtual machine state
  • Memory over-commitment leading to ESXi swapping or VM memory ballooning
  • Sharing disks between virtual machines without a clustering solution may lead to data corruptions
  • Hot Extend Local VMDK file even it is not associated to SCSI bus sharing controller.

NOTE: For more information on VM configuration limitations, see the vSphere WSFC Setup Limitation section in the vSphere Resource Management Guide.


Microsoft support policies for a virtualized deployment of WSFC

Microsoft support deployment of WSFC on a VM. Check the Microsoft SVVP Program for more details Windows Server Virtualization Validation Program . Also check the Microsoft KB article Support policy for Microsoft SQL Server products that are running in a hardware virtualization environment .

Workaround:
 


Additional Information

Disclaimer: VMware is not responsible for the reliability of any data, opinions, advice, or statements made on third-party websites. Inclusion of such links does not imply that VMware endorses, recommends, or accepts any responsibility for the content of such sites.

简体中文：VMware vSphere 6.x 上的 Microsoft Windows Server 故障切换群集：支持的配置指南
VMware vSphere 6.x での Microsoft Windows Server フェイルオーバ クラスタ：サポートされる構成のためのガイドライン

Powered by Wolken Software