VKS Cluster does not provision with the error: failed to cache files
search cancel

VKS Cluster does not provision with the error: failed to cache files

book

Article ID: 446755

calendar_today

Updated On:

Products

VCF Private AI Services VMware Cloud Foundation

Issue/Introduction

VKS Cluster does not provision.

VM operator logs report the error: failed to cache files: failed to cache storage items: failed to wait for copy .vmdk: A specified parameter was not correct: fileType

Warning CreateFailure 15s (x14 over 57s) vmware-system-[MASKED ID]/vmware-system-[MASKED ID]/virtualmachine-controller failed to cache files: failed to cache storage items: failed to wait for copy . vmdk: A specified parameter was not correct: fileType

vCenter Server logs report the error: vmodl.fault.InvalidArgument

Log File: /var/log/vmware/vpxd/vpxd.log

YYYY-MM-DDTHH:MM:SS.MS+HH:MM ERROR vpxd 5743 [vc@4413 sub="Default" opID="432d1###"] [VpxLRO] -- ERROR task-48022 -- ########-####-####-####-############ (########-####-####-####-############) - virtualdiskManager -- vim.VirtualDiskManager.copyVirtualDisk: : vmodl.fault.InvalidArgument
-->Result:
--> (vmodl.fault.InvalidArgument) {
-->     faultCause= (vmodl.MethodFault) null,
-->     faultMessage = <unset>,
-->     invalidproperty = "fileType"
-->     msg = ""
--> }
-->
--> Args:
--> Arg sourceName:
- ->"[vsanDatastore] ########-####-####-####-############/########-####-####-####-############/ubuntu-ova-disk1_#######-####-###-###-##########.vmdk"
--> Arg sourceDatacenter:
-->'vim.Datacenter: datacenter-#'
--> Arg destName:
--> "[vsanDatastore] vmi-############-############-############ / ############.vmdk"
--> Arg destDatacenter:
-->"vim.Datacenter: datacenter-#'
--> Arg destspec:
-->(vim VirtualDiskManager-FileBackedVirtualDiskSpec) {
-->    diskType = "thin"

Content Library Logs report the error: Unexpected end of file and Read timed out.

Log file: /var/log/vmware/content-library/cls.log

YYYY-MM-DDTHH:MM:SS.MS+HH:MM ERROR CLS 495 [vc@4413 threadName="transferService-nfc-##############-ubuntu-ova-disk1.vmdk-upload" logger="NfcEndpointImpl"]  Session ##############: Unable to parse disk image while uploading to: ds:///vmfs/volumes/[MASKED DS]//contentlib-########-####-####-####-############/########-####-####-####-############/ubuntu-ova-disk1_#######-####-####-####-##########.vmdk?serverId=########-####-####-####-############
com.vmware.transfer.streamVmdk.VmdkFormatException: Unexpected end of file
          at com.vmware.transfer.streamVmdk.StreamVmdkReader.readFully(StreamVmdkReader.java:241)
          at com.vmware.transfer.streamVmdk.StreamVmdkReader.getNextGrain(StreamVmdkReader.java:217)
          at com.vmware.transfer.impl.NfcEndpointImpl$DiskUploader.attemptDiskUpload(NfcEndpointImpl.java:1179)
          at com.vmware.transfer.impl.NfcEndpointImpl$DiskUploader.upload(NfcEndpointImpl.java:1141)
          at com.vmware.transfer.impl.NfcEndpointImpl$1.run(NfcEndpointImpl.java:195)
Caused by: java.io.EOFException
          at java.base/java.io.DataInputStream.readFully(Unknown Source)
          at java.base/java.io.DataInputStream.readFully(Unknown Source)
          at com.vmware.transfer.streamVmdk.StreamVmdkReader.readFully(StreamVmdkReader.java:239)
          ... 4 more

YYYY-MM-DDTHH:MM:SS.MS+HH:MM DEBUG CLS 490 [vc@4413 threadName="transferservice-pool-#-thread-##" logger="TransferItemImpl"]
##############: no transfer scheduled due to source error for item ubuntu-ova-disk1.vmdk: IO error during transfer of https://wp-content.vmware.com/v2/latest/ob-######-ubuntu-####-amd64-v1.##.#--vmware.1-fips-vkr.2/ubuntu-ova-disk1.vmdk: Read timed out.

Environment

VMware Cloud Foundation 5.x

VMware Cloud Foundation 9.x

Cause

The cluster provisioning process relies on VMDK assets stored in the content library. The content library subscribes to VMware Kubernetes releases published on VMware's Content Delivery Network .

The issue occurs because the content library is unable to download these required assets from https://wp-content.vmware.com/v2/latest/

Resolution

Review and validate that the vCenter Server is able to access the content delivery site, https://wp-content.vmware.com/v2/latest/

As a workaround, a local content library can be configured. See Create a Local Content Library (for Air-Gapped Cluster Provisioning)

Additional Information

Content library sync fails with error "IO error while uploading to datastore"