How to configure number of CPU sockets for BOSH created VM in TAS on vSphere environment

Products

• VMware Tanzu Application Service for VMs

Issue/Introduction

Sometimes users might want specific number of CPU sockets for BOSH created VM on vSphere for special purpose (e.g. maximizing the performance). But when they try to add more CPUs by changing the VM type, it ends up adding socket not core. So, they may want to fix the number of sockets so that whenever they are scaling VM, it's only to increase core not socket.
For example, a VM of large.cpu type would have 4 sockets with 1 core each with current implementation.

- cloud_properties:
 cpu: 4
 disk: 16384
 ram: 4096
 name: large.cpu

$ egrep 'physical id|core|processor' /proc/cpuinfo
processor : 0
physical id : 0
core id : 0
cpu cores : 1
processor : 1
physical id : 2
core id : 0
cpu cores : 1
processor : 2
physical id : 4
core id : 0
cpu cores : 1
processor : 3
physical id : 6
core id : 0
cpu cores : 1

Environment

Product Version: 2.12

Resolution

As described in this KB article and illustrated by the example in the BOSH documentation , the cloud property cpuid.coresPerSocket could help to determine number of CPU socket on a VM. Specifically, it is <number of CPU socket> = <number of CPU>/cpuid.coresPerSocket. For example, manually adding a new VM type with following spec to existing cloud config, and create a VM with such type, you could get a VM with 2 sockets and 2 cores for each socket.

- cloud_properties:
 cpu: 4
 disk: 8192
 ram: 512
 vmx_options:
   cpuid.coresPerSocket: "2"
 name: s2.c4

Instance                                         VM Type
testvm/4d58069d-8019-49c5-a63e-f7268b615ea0      s2.c4

testvm/4d58069d-8019-49c5-a63e-f7268b615ea0:~$ egrep 'physical id|core|processor' /proc/cpuinfo
processor : 0
physical id : 0
core id : 0
cpu cores : 2
processor : 1
physical id : 0
core id : 1
cpu cores : 2
processor : 2
physical id : 1
core id : 0
cpu cores : 2
processor : 3
physical id : 1
core id : 1
cpu cores : 2

However manual update to existing cloud config is not persistent. It will be overwritten by next “Apply Changes” through Ops Manager. Besides, the operator is unable to pick up such new VM type on Ops Manager web UI for product job on Resource Config pane. Though Ops Manager provides API endpoint /api/v0/vm_types for adding customized VM type, but it doesn’t support vmx_options property.  A feature request has been submitted to product team for supporting vmx_options property by endpoint /api/v0/vm_types.

Before the feature is developed, one possible solution is to utilize custom VM extension as illustrated by below example.

1) Add a VM extension with desired number of cores per socket

$ uaac curl -k "https://<Ops Manager FQDN>/api/v0/staged/vm_extensions/4cores_per_socket" \
 -X PUT \
 -H "Content-Type: application/json" \
 -d '{"name": "4cores_per_socket", "cloud_properties": {"vmx_options": {
 "cpuid.coresPerSocket": "4"}}}'

2) Apply the new exension to some existing job

- cloud_properties:
 cpu: 8
 disk: 32768
 ram: 8192
 name: xlarge.cpu

$ uaac curl -k "https://<Ops Manager FQDN>/api/v0/staged/products/cf-xxxx/jobs/backup_restore-yyyy/resource_config" \
 -X PUT \
 -H "Content-Type: application/json" \
 -d '{
 "instance_type": {
 "id": "xlarge.cpu"
 },
 "additional_vm_extensions": ["4cores_per_socket"]
}'

3) Do “Apply Changes” through Ops Manager against the related product tile. The VMs for the job will be recreated to apply the VM extension

backup_restore/35f242d7-6b7c-476e-9e0b-eab183fbbacc:~$ egrep 'physical id|core|processor' /proc/cpuinfo
processor : 0
physical id : 0
core id : 0
cpu cores : 4
processor : 1
physical id : 0
core id : 1
cpu cores : 4
processor : 2
physical id : 0
core id : 2
cpu cores : 4
processor : 3
physical id : 0
core id : 3
cpu cores : 4
processor : 4
physical id : 1
core id : 0
cpu cores : 4
processor : 5
physical id : 1
core id : 1
cpu cores : 4
processor : 6
physical id : 1
core id : 2
cpu cores : 4
processor : 7
physical id : 1
core id : 3
cpu cores : 4

NOTE: Make sure <number of CPU socket> = <number of CPU>/cpuid.coresPerSocket will yield a integer number. For example, if you configure a VM with 20 vCPUs and coresPerSocket to 16, which means <number of CPU socket> = 20/16 = 1.25,  then you might encounter error (as show below) when applying the changes.

Error: Unknown CPI error 'Unknown' with message 'Module CPUID power on failed. ' in 'create_vm' CPI method (CPI request ID: 'cpi-134331')