VMware Cloud Foundation fails to claim vSAN Disks on a host using all flash storage when creating a workload domain
Article ID: 316030
Updated On:
Products
VMware Cloud Foundation
Issue/Introduction
Symptoms:
- Creating a workload domain fails with the error similar to:
"Failed to claim disk for vSAN".
- In the /home/vrack/vrm/logs/vrack-vrm-debug.log file on the SDDC Manager Controller virtual machine, you see an activity and errors messages similar to:
2018-01-25 19:25:17.279 [pool-26-thread-9] DEBUG [com.vmware.vrack.vrm.workflow.tasks.vsan.ClaimHostDisksVsanTask] <<< Claiming host 192.168.100.116 disks TestVI
2018-01-25 19:25:17.287 [pool-26-thread-9] INFO [com.vmware.vrack.vrm.vsan.VsanServiceImpl] Start to claim host ManagedObjectReference: type = HostSystem, value = host-69, serverGuid = <uuid> disks in VSAN cluster
2018-01-25 19:25:17.311 [pool-26-thread-9] INFO [com.vmware.vrack.vrm.vsan.VsanServiceImpl] Adding eligible SSD disk naa.68ce38ee3005a5f9 for VSAN
2018-01-25 19:25:17.311 [pool-26-thread-9] INFO [com.vmware.vrack.vrm.vsan.VsanServiceImpl] Adding eligible SSD disk naa.6002538a675a3460 for VSAN
2018-01-25 19:25:17.311 [pool-26-thread-9] INFO [com.vmware.vrack.vrm.vsan.VsanServiceImpl] Adding eligible SSD disk naa.6002538a675a34a0 for VSAN
2018-01-25 19:25:17.311 [pool-26-thread-9] INFO [com.vmware.vrack.vrm.vsan.VsanServiceImpl] Adding eligible SSD disk naa.6002538a675a34b0 for VSAN
2018-01-25 19:25:17.311 [pool-26-thread-9] INFO [com.vmware.vrack.vrm.vsan.VsanServiceImpl] Adding eligible SSD disk naa.6002538a675a41e0 for VSAN
2018-01-25 19:25:17.311 [pool-26-thread-9] INFO [com.vmware.vrack.vrm.vsan.VsanServiceImpl] Adding eligible SSD disk naa.6002538a675a3040 for VSAN
2018-01-25 19:25:17.311 [pool-26-thread-9] INFO [com.vmware.vrack.vrm.vsan.VsanServiceImpl] Skipping disk t10.ATA_____SATADOM2DML_3SE__________________________TW00T4M44777777I0403 for VSAN: State: ineligible
2018-01-25 19:25:17.311 [pool-26-thread-9] INFO [com.vmware.vrack.vrm.vsan.VsanServiceImpl] Adding eligible SSD disk naa.6002538a675a4250 for VSAN
2018-01-25 19:25:17.312 [pool-26-thread-9] INFO [com.vmware.vrack.vrm.vsan.VsanServiceImpl] Adding eligible SSD disk naa.68ce38ee3005a601 for VSAN
2018-01-25 19:25:17.312 [pool-26-thread-9] INFO [com.vmware.vrack.vrm.vsan.VsanServiceImpl] Adding eligible SSD disk naa.6002538a675a2520 for VSAN
2018-01-25 19:25:17.312 [pool-26-thread-9] INFO [com.vmware.vrack.vrm.vsan.VsanServiceImpl] Adding eligible SSD disk naa.6002538a675a4260 for VSAN
2018-01-25 19:25:17.312 [pool-26-thread-9] INFO [com.vmware.vrack.vrm.vsan.VsanServiceImpl] No NonSSD disks found eligible for VSAN. Trying to use SSD disks as capacity (allFlash mode)
2018-01-25 19:25:17.312 [pool-26-thread-9] INFO [com.vmware.vrack.vrm.vsan.VsanServiceImpl] Getting host 192.168.100.116 disks tagged as CapacityFlash
2018-01-25 19:25:17.312 [pool-26-thread-9] DEBUG [com.vmware.vrack.vrm.core.util.SshSession] Prepare ssh session configuration properties.
2018-01-25 19:25:17.312 [pool-26-thread-9] DEBUG [com.vmware.vrack.vrm.core.util.SshSession] Prepared ssh session configuration properties with strict host checking: true.
2018-01-25 19:25:17.329 [pool-26-thread-9] INFO [com.vmware.vrack.vrm.core.util.SshSession] Connecting to 192.168.100.116...
2018-01-25 19:25:17.431 [pool-26-thread-9] INFO [com.vmware.vrack.vrm.core.util.SshSession] Connected to 192.168.100.116
2018-01-25 19:25:17.431 [pool-26-thread-9] INFO [com.vmware.vrack.vrm.vsan.VsanServiceImpl] Connected to: 192.168.100.116
2018-01-25 19:25:17.431 [pool-26-thread-9] DEBUG [com.vmware.vrack.vrm.core.util.SshUtil] Starting to execute command [cat /store/.capacityflash.json]
2018-01-25 19:25:17.974 [pool-26-thread-9] DEBUG [com.vmware.vrack.vrm.core.util.SshUtil] End of execution of command [cat /store/.capacityflash.json] status : 1 ,
Output:
Error: [******c******a******t******:****** ******c******a******n******'******t****** ******o******p******e******n****** ******'******/******s******t******o******r******e******/******.******c******a******p******a******c******i******t******y******f******l******a******s******h******.******j******s******o******n******'******:****** ******N******o****** ******s******u******c******h****** ******f******i******l******e****** ******o******r****** ******d******i******r******e******c******t******o******r******y******
******]
2018-01-25 19:25:17.974 [pool-26-thread-9] ERROR [com.vmware.vrack.vrm.core.util.SshUtil] SSH Execution of command cat /store/.capacityflash.json : Failed (Exit Code 1 ) :
2018-01-25 19:25:17.975 [pool-26-thread-9] INFO [com.vmware.vrack.vrm.vsan.VsanServiceImpl] Disconnected from: 192.168.100.116
2018-01-25 19:25:17.975 [pool-26-thread-9] ERROR [com.vmware.vrack.vrm.workflow.tasks.AbstractVsanTask] Virtual SAN: Claim Hosts Disks [ClaimHostDisksVsanTask]: failed. Failed to get vSAN disks capacity flash info.
2018-01-25 19:25:17.976 [pool-26-thread-9] ERROR [com.vmware.vrack.vrm.core.engine.SimpleWorkflowEngine] Exception in task: Virtual SAN: Claim Hosts Disks for Workflow <uuid>
com.vmware.vrack.vrm.workflow.tasks.TaskException: Failed to get vSAN disks capacity flash info.
at com.vmware.vrack.vrm.workflow.tasks.AbstractVsanTask.run(AbstractVsanTask.java:67) ~[vrm-iaas-2.2.1-RELEASE.jar:na]
at com.vmware.vrack.vrm.core.engine.SimpleWorkflowEngine.executeTask(SimpleWorkflowEngine.java:539) [vrm-core-2.2.1-RELEASE.jar:na]
at com.vmware.vrack.vrm.core.engine.SimpleWorkflowEngine.runSingleTask(SimpleWorkflowEngine.java:493) [vrm-core-2.2.1-RELEASE.jar:na]
at com.vmware.vrack.vrm.core.engine.SimpleWorkflowEngine.access$1000(SimpleWorkflowEngine.java:47) [vrm-core-2.2.1-RELEASE.jar:na]
at com.vmware.vrack.vrm.core.engine.SimpleWorkflowEngine$2.run(SimpleWorkflowEngine.java:304) [vrm-core-2.2.1-RELEASE.jar:na]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [na:1.8.0_141]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [na:1.8.0_141]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_141]
Caused by: com.vmware.vrack.vrm.vsan.VsanException: Failed to get vSAN disks capacity flash info.
at com.vmware.vrack.vrm.vsan.VsanServiceImpl.getHostCapacityFlashDisks(VsanServiceImpl.java:963) ~[vrm-vsphere-2.2.1-RELEASE.jar:na]
at com.vmware.vrack.vrm.vsan.VsanServiceImpl.claimHostDisks(VsanServiceImpl.java:907) ~[vrm-vsphere-2.2.1-RELEASE.jar:na]
at com.vmware.vrack.vrm.workflow.tasks.vsan.ClaimHostDisksVsanTask.doInVsan(ClaimHostDisksVsanTask.java:45) ~[vrm-iaas-2.2.1-RELEASE.jar:na]
at com.vmware.vrack.vrm.workflow.tasks.AbstractVsanTask.run(AbstractVsanTask.java:62) ~[vrm-iaas-2.2.1-RELEASE.jar:na]
... 7 common frames omitted
Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.
Environment
VMware Cloud Foundation 4.x
Cause
In order for VMware Cloud Foundation to successfully claim the disks in an all flash configuration, the file /store/.capacityflash.json needs to be present and accessible. This file is created during imaging only if the all flash option is manually selected during the process. If not, it is assumed that the vSAN will be configured in hybrid mode, by default, and the file will not be generated. Alternatively, if the storage is not present at the time of provisioning, even if all flash mode is specified, the file may not be generated.
Resolution
To resolve this issue, perform one of the below options:
Option 1
Re-image the host and specify the all flash mode option during the process. Ensure that the storage is present and configured correctly on the host before starting the provisioning process.
Option 2
The /store/.capacityflash.json file can be manually created using the following procedure:
- Connect to the host as root using the console or SSH.
- From the command line, mark the flash disks being used for capacity as capacity disks:
vSphere 6.0
vSphere 6.5
- Change to the /store directory:
cd /store
- Confirm that the .capacityflash.json does not already exist.
ls -la
- Create the .capacityflash.json file and populate it with the disk information from vdq -q
vdq -q > .capacityflash.json
You should now see a listing similar to the following:
-rwx------ 1 root root 743 Jan 30 18:54 .capacityflash.json
- Adjust the content created by the previous command to accommodate the expected formatting. Using a text editor (For Example: vi), perform the following changes:
- Remove the comma after the final attribute in each section. For example, the following section was created by the vdq command:
{
"Name" : "naa.6589cfc000000100085fce5452881f4f",
"VSANUUID" : "",
"State" : "Ineligible for use by VSAN",
"Reason" : "Has partitions",
"IsSSD" : "1",
"IsCapacityFlash": "0",
"IsPDL" : "0",
},
Once this is corrected, it will be similar to the following:
{
"Name" : "naa.6589cfc000000100085fce5452881f4f",
"VSANUUID" : "",
"State" : "Ineligible for use by VSAN",
"Reason" : "Has partitions",
"IsSSD" : "1",
"IsCapacityFlash": "0",
"IsPDL" : "0"
},
- Remove the comma after the final section before the closing square bracket (i.e., the ] at the end of the file). For example, the following section is the format of the final section after making the change specified above:
{
"Name" : "naa.6589cfc000000100085fce5452881f4f",
"VSANUUID" : "",
"State" : "Ineligible for use by VSAN",
"Reason" : "Has partitions",
"IsSSD" : "1",
"IsCapacityFlash": "0",
"IsPDL" : "0"
},
]
After removing the trailing comma at the end of the section, it will appear as follows:
{
"Name" : "naa.6589cfc000000100085fce5452881f4f",
"VSANUUID" : "",
"State" : "Ineligible for use by VSAN",
"Reason" : "Has partitions",
"IsSSD" : "1",
"IsCapacityFlash": "0",
"IsPDL" : "0"
}
]
- Remove the comma after the final attribute in each section. For example, the following section was created by the vdq command:
- Retry the failed workflow.
Feedback
Yes
No
Powered by
