What You Can (and Cannot) Change in a vSAN ReadyNode™
Article ID: 326476
Updated On:
Products
Issue/Introduction
Overview
VMware vSAN has taken the Hyper-Converged Infrastructure (HCI) market by storm. More than 30,000 customers have now adopted vSAN with thousands of deployments, confirming its position as the leading product in hyper-converged infrastructure (HCI).
One of the consumption models of vSAN software is ReadyNodes and today we have more than 200 ReadyNodes listed in vSAN VMware Compatibility Guide (VCG) from OEM partners across the globe.
So what are vSAN ReadyNodes?
vSAN ReadyNodes are x86 servers, available from all the leading server vendors, that have been pre-configured, tested, and certified for VMware Hyper-Converged Infrastructure Software. Each ReadyNodeTM is optimally configured for vSAN with the required amount of CPU, memory, network, I/O controllers, and storage (SSDs, HDDs, or flash devices). Each ReadyNodeTM listed in vSAN VCG has an SKU. You can enter that SKU into OEM partners’ system configurator and it will generate a BOM that is “flexible".
We often get asked by our field, partners, and customers that “Can a ReadyNodeTM build of materials (BOM) be changed?” e.g. can we use another capacity tier device or use a better processor than listed in the BOM. The short answer is “Yes” – there are things in a vSAN ReadyNodeTM BOM that you can change and there are things you cannot.
Resolution
To provide guidelines for what can be changed in a ReadyNodeTM, we are taking a two-phased approach.
- Provide guidance to customers, partners, and field on what we allow to modify within a vSAN ReadyNodeTM BOM today by means of this blog which will be linked to the official vSAN VMware Compatibility Guide (OSA) or vSAN VMware Compatibility Guide (ESA)
- Subsequently, build a tool that enables customers, partners, and field to select changed configuration automatically and create in runtime a vSAN ReadyNodeTM BOM from a supported list of components on the vSAN VCG.
In this blog, I will outline the first approach “Guidance on which the tool is going to be built upon”.
To begin with, below is an example for vSAN ReadyNodeTM BOM in vSAN VCG:
Sample vSAN ReadyNodeTM BOM
If we look at components in vSAN ReadyNodeTM BOM, most of them are allowed to be changed based on some guidelines and the below table describes those guidelines.
Memory: Adding more memory than what is listed is supported. Please maintain a balanced memory population configuration where possible.
Allowable Changes in vSAN ReadyNodeTM BOM
For your reference, caching tier guidelines can be read here Design Considerations for Flash Caching Devices in vSAN & Understanding vSAN Architecture: Disk Groups, and boot device KB vSphere SSD and Flash Device Support.
As you can see, vSAN ReadyNodeTM BOM is flexible enough to accommodate the change as long as the component is listed on the vSAN VCG (ESA), vSAN VCG (OSA), or vSphere VCG, as applicable.
Before I discuss further on storage sub-systems items, I would like to highlight that each ReadyNodeTM server platform is built on a “standard x86 server platform” that needs to be ESXi (vSphere) certified. This is a pre-requisite for ReadyNodeTM listing.
Expanding on the above thoughts, I would like to highlight the flexibility we have on storage sub-systems of vSAN ReadyNodeTM BOM as shown below:
Allowable Changes in Storage Sub-Systems of vSAN ReadyNodeTM BOM
Note: Performance characteristics of the ReadyNodeTM may vary if you change any component.
Lastly, I have put some FAQs which will help address few additional areas on that.
vSAN ReadyNodeTM Storage Component FAQ:
Q: - Can I choose a different make and model of cache and capacity tier drive vSAN ReadyNodeTM BOM?
Please refer to the table above for guidelines.
Q: - Can I replace HDD with SSD in the capacity tier in vSAN ReadyNodeTM ?
No. vSAN ReadyNodeTM profile is meant for AF or HY. You cannot mix and match.
Q: - Why can’t I use a different IO controller than listed in ReadyNodeTM ?
IO controller is one of the key components in vSAN ReadyNodeTM profile performance guidelines. We run an extensive performance test on IO controller before it is certified in a system. I/O controller pretty much defines how much I/O can be pushed down to the underlying storage subsystem and a substandard controller that is not certified for vSAN can result in performance and even availability issues.
Q: - Can I move from a lower vSAN ReadyNodeTM profile to a higher by changing the configuration?
Yes, you are allowed to move to a higher configuration based on your IOPS and capacity needs. This guidance is primarily meant for helping to create flexibility within a vSAN ReadyNodeTM.
vSAN ReadyNodeTM configurator is a lightweight tool with minimum inputs. BOM customization will be a part of vSAN sizer.
Q: - Do we need Boot Device Certification?
We do not need to certify the boot device for vSAN. Please follow the guidelines laid out in this KB vSphere SSD and Flash Device Support.
Q: - Do we need to certify the NIC in a ReadyNodeTM BOM?
There is no separate NIC certification required for vSAN. NIC cards qualified under IOVP certification are used for vSAN ReadyNodeTM BOM.
Q: - Do we use Expander or add a new Controller for more drives.
It depends. If the workload needs storage dense configuration, we recommend using an expander. When the workload demands more performance or capacity beyond the number of drives tested for the expander, adding certified controllers and configuring disk groups behind the added controllers may help.
e.g. a P440 controller tested and certified up to 16 drives. If you need to configure up to 24 drives, you can put 16 drives beyond an expander and the additional 8 (1 + 7) drives (by adding a new disk group) beyond a new certified controller. It will provide additional storage as well as increased performance for the ReadyNodeTM .
Additionally, some systems are built with an onboard expander and you may not need to put additional expander but you can add a controller in such a scenario for better performance.
We allow expander support only on ReadyNodeTM. If you are building your own (BYO) vSAN, we do not support the expander.
Q: - Can we use M.2 devices for vSAN?
We see this device getting more attention as a mirrored boot device configuration in a vSAN environment. For now, our guidance is to use mirrored M.2 as boot device connected to a controller in RAID 1 mode (which is different from the primary controller used for the vSAN datastore). This allows for high availability and self-contained storage for logs, traces, core dumps etc in addition to using it as a boot device. Putting the mirrored M.2s behind a controller separate from the controller behind which the vSAN datastore is a best practice that we highly recommend.
In the future, we will look into this form factor for other areas.
Q:- What can be changed when we say "Adding more memory is allowed"?
You are allowed to do the following changes:
- You can put more memory than listed in the ReadyNodeTM.
- You can choose different DIMM size (16GB, 32 GB, etc...) to meet your memory requirement(Please do not mix DIMM sizes in the same platform).
- You can choose the same or higher memory frequency than listed as part of the BOM.
- Please work with your OEM vendor for an optimal configuration.
Please note that some applications are memory intensive and may require the exact specifications to be met. In such cases, the application owner needs to identify the exact requirement. Our general guidance is you are allowed to modify the above points and you will not require any additional certification or new ReadyNodeTM listing for belonging to the profile.
Q:- Can we swap my 512n HDD device with 512e or vice versa in a ReadyNodeTM?
Yes, you can. If the ReadyNodeTM is listed 512e and swapped with 512n, you are expected to have better performance based on your workload. However, you may experience, especially with small block size workload, performance degradation when 512n is swapped with 512e HDD drives.
Q:- What is vSAN Secure-wipe capable ReadyNodeTM ?
vSAN SecureWipe capable ReadyNodeTM is a ReadyNodeTM whose components (IO Controllers, Expanders, SAS/SATA/NVMe SSD Drives) are certified for Disk Sanitization.
Q:- Do we support the BYO scenario for Secure-wipe?
No, vSAN Secure-Wipe capable feature is not supported for the BYO scenario.
Q. What is vSAN support for PMem?
Please refer to KB vSphere/vSAN Support for Intel's Optane Persistent Memory (Pmem) for vSAN support of PMem in Memory Mode.
Q:- Is 4S Sapphire Rapids supported in OSA?
Sapphire Rapids 4S servers are not supported for BYO at this time.
Q:- Can I leverage a 2-socket server for a single socket configuration?
Yes, reducing to single socket config is supported but the total core counts must remain the same
For any questions on vSAN hardware, please reach out to vSAN Hardware PM at [email protected].
To learn more about vSAN, visit VMware vSAN
Additional Information
Feedback
