Symptoms:
When connected to vCenter Server, you experience these symptoms:
- Cannot create virtual machines from templates.
- Cannot create a new virtual machine.
- You see the alarm:
Thin-provisioned storage devices met soft threshold
- The storage utilization on a LUN is above 90%.
- In the
vpxd.log
file, you see entries similar to:
mem> [verbose 'Default' opID=14603DCE-00001015-3f] [VpxVmomi] Invoke done: vim.Folder.createVm session: f7cc8b40-671c-4729-02e6-9a12996fc5b8
mem> [verbose 'SoapAdapter.HTTPService' opID=14603DCE-00001015-3f] HTTP Response: Complete (processed 495 bytes)
mem> [verbose 'PropertyProvider'] RecordOp ASSIGN: info.startTime, task-29974
mem> [verbose 'Default' opID=14603DCE-00001013-22] [VpxVmomi] Invoke done: vmodl.query.PropertyCollector.waitForUpdates session: f7cc8b40-671c-4729-02e6-9a12996fc5b8
mem> [verbose 'PropertyProvider' opID=14603DCE-00001015-3f] RecordOp ASSIGN: info.state, task-29974
mem> [verbose 'PropertyProvider' opID=14603DCE-00001015-3f] RecordOp ASSIGN: info.cancelable, task-29974
mem> [verbose 'SoapAdapter.HTTPService' opID=14603DCE-00001013-22] HTTP Response: Complete (processed 1390 bytes)
mem> [verbose 'Default' opID=86defaf9] [VpxVmomi] Invoking [waitForUpdatesEx] on [vmodl.query.PropertyCollector:propertyCollector] session [f832c9de-dad7-37c8-88e9-0d2469a97af4(52b78214-ccc8-4a0a-e7a8-bbccdc34fdac)]
mem> [verbose 'SSL SoapAdapter.HTTPService.HttpConnection'] User agent is 'VMware vim-java 1.0'
mem> [verbose 'Default' opID=934f3ba3] [VpxVmomi] Invoking [waitForUpdatesEx] on [vmodl.query.PropertyCollector:propertyCollector] session [e4259a71-363b-a3a2-2d45-cc700f98a79f(5205ee3e-f96d-fc0c-a29a-71d3dde095cf)]
mem> [verbose 'SoapAdapter.HTTPService.HttpConnection'] User agent is 'VMware VI Client/4.0.0'
mem> [info 'commonvpxLro' opID=14603DCE-00001015-3f] [VpxLRO] -- BEGIN task-29974 -- group-v22 -- vim.Folder.createVm -- f7cc8b40-671c-4729-02e6-9a12996fc5b8(52670acd-0070-f3fa-78c7-e23c62b329ba)
mem> [verbose 'SoapAdapter.HTTPService.HttpConnection'] User agent is 'VMware VI Client/4.0.0'
mem> [verbose 'vpxdvpxdMoDatastore' opID=14603DCE-00001015-3f] [IsDatastoreActive] datastore DATASTORE_VMFS01 disabled (Low free space)
mem> [error 'VmProv' opID=14603DCE-00001015-3f] [CreateDestDatastores] moDs is not active for DATASTORE_VMFS01
mem> [error 'Default' opID=14603DCE-00001015-3f] (Log recursion level 2) vim.fault.InvalidDatastoreState
------ In-memory logs end --------
[error 'Default' opID=14603DCE-00001015-3f] Section for VMware VirtualCenter, pid=3924, version=5.1.0, build=947673, option=Release
[info 'commonvpxLro' opID=14603DCE-00001015-3f] [VpxLRO] -- FINISH task-29974 -- group-v22 -- vim.Folder.createVm --
[info 'Default' opID=14603DCE-00001015-3f] [VpxLRO] -- ERROR task-29974 -- group-v22 -- vim.Folder.createVm: vim.fault.InvalidDatastoreState:
--> Result:
--> (vim.fault.InvalidDatastoreState) {
--> dynamicType = <unset>,
--> faultCause = (vmodl.MethodFault) null,
--> datastoreName = "DATASTORE_VMFS01",
--> msg = "",
--> }
--> Args:
[info 'commonvpxLro' opID=14603DCE-00001016-55] [VpxLRO] -- BEGIN task-internal-6918 -- -- vmodl.query.PropertyCollector.createFilter -- f7cc8b40-671c-4729-02e6-9a12996fc5b8(52670acd-0070-f3fa-78c7-e23c62b329ba)
[info 'commonvpxLro' opID=14603DCE-00001016-55] [VpxLRO] -- FINISH task-internal-6918 -- -- vmodl.query.PropertyCollector.createFilter --
[info 'commonvpxLro' opID=14603DCE-00001019-d5] [VpxLRO] -- BEGIN task-internal-6919 -- -- vmodl.query.PropertyCollector.Filter.destroy -- f7cc8b40-671c-4729-02e6-9a12996fc5b8(52670acd-0070-f3fa-78c7-e23c62b329ba)
[info 'commonvpxLro' opID=14603DCE-00001019-d5] [VpxLRO] -- FINISH task-internal-6919 -- -- vmodl.query.PropertyCollector.Filter.destroy --
[info 'commonvpxLro' opID=HB-host-64@2686-7be78501] [VpxLRO] -- BEGIN task-internal-6920 -- host-64 -- VpxdInvtHostSyncHostLRO.Synchronize --
[info 'vpxdvpxdInvtHostCnx' opID=HB-host-64@2686-7be78501] [VpxdHostSync] Synchronizing host: host-64 (hostname.dom.com, 10.118.1.234)
[info 'vpxdvpxdVmomi'] [ClientAdapterBase::InvokeOnSoap] Invoke done (hostname.dom.com, vmodl.query.PropertyCollector.waitForUpdates)
[info 'vpxdvpxdVmomi' opID=HB-host-64@2686-7be78501] [ClientAdapterBase::InvokeOnSoap] Invoke done (hostname.dom.com, vpxapi.VpxaService.retrieveChanges
Note: All the log messages starting with mem> are from a memory dump. Similar looking lines will be in vpxd.log but they wont start with mem>.
Note: This article describes a specific issue. If you experience all of the preceding symptoms on a particular datastore, refer the following sections. If you experience some but not all of these symptoms, your issue is not related to this article. Search for your symptoms in the Knowledge Base or
Open a Support Request.