"Negative uncommittedCapacityMb" error when uploading a file to a Cloud Director catalog

Products

• VMware Cloud Director

Issue/Introduction

• The 'Imported Media File' task fails with the error 'Negative uncommittedCapacityMb: -8796093022208'
• Within the log file /opt/vmware/vcloud-director/logs/vcloud-container-debug you see the following errors:

2022-08-30 12:24:59,384 | ERROR  | Backend-activity-pool-2748 | ComposeRecomposeVAppActivity  | [Activity Execution] Error during CreatedItemsProcessingPhase - Handle: urn:uuid:<HANDLE_UUID>, Current Phase: ComposeRecomposeVAppActivity$CreatedItemsProcessingPhase | requestId=<REQUEST_UUID>,request=POST https://<VCD_URL>/api/vdc/<VDC_UUID>/action/createVm,requestTime=1630326295995,remoteAddress=<REMOTE_ADDRESS>:50902,userAgent=Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 ...,accept=application/*+xml;version 37.0.0-alpha vcd=<VCD_UUID>,task=<TASK_UUID> activity=(com.vmware.vcloud.backendbase.management.system.TaskActivity,urn:uuid:<TASK_UUID>) activity=(com.vmware.vcloud.vdc.impl.ComposeRecomposeVAppActivity,urn:uuid:<ACTIVITY_UUID>)
java.lang.IllegalArgumentException: Negative uncommittedCapacityMb: -8796093022208
    at com.vmware.vcloud.fabric.storage.placement.algorithm.StoragePlacementAlgorithm$Container.<init>(StoragePlacementAlgorithm.java:420)
    at com.vmware.vcloud.fabric.storage.placement.algorithm.StoragePlacementAlgorithmUtils$ParamsBuilder.addContainer(StoragePlacementAlgorithmUtils.java:158)

Caused by: org.hibernate.HibernateException: org.hibernate.exception.DataException: could not load an entity: [com.vmware.vcloud.common.model.inventory.DatastoreInvModel#########-####-####-####-#######]

• vCenter Server may become disconnected from Cloud Director.

Environment

VMware Cloud Director 10.x

Cause

This is a known issue caused when vSphere 7.0 Update 2 generates incorrect data regarding storage metrics. This data is then transmitted to Cloud Director and can result in the error 'Negative uncommittedCapacityMb' being visible in Cloud Director during certain tasks. For additional information see vSphere KB 83990.

Resolution

The issue is resolved in ESXi 7.0 Update 2c (build number 18426014) which is available for download here.

Note: Cloud Director 10.3.1 and above also resolves the issue whereby vCenter Server may disconnect as a result of this problem. However that the 'Negative uncommittedCapacityMb' error can still be visible until the ESXi patch is applied.

Workaround:

• Within vCenter Server, perform a full storage rescan from all ESXi clusters backing the Provider VDC in order to refresh the storage metrics. Allow a few minutes for this data to synchronize in Cloud Director.