[YYYY-MM-DDTHH:MM:SS] [ERROR] http-nio-5090-exec-1525 71189500 113157 200764 c.v.v.client.vm.storageDrs.impl.StorageRecommendationsValidator com.vmware.vim.vmomi.client.exception.ConnectionException: http://localhost:1080/external-vecs/http2/Test.test.test.s.com/443/sdk invocation failed with "java.net.SocketTimeoutException: 600,000 milliseconds timeout on connection http-outgoing-28382 [ACTIVE]"]
[YYYY-MM-DDTHH:MM:SS] [ERROR] http-nio-5090-exec-1525 71189500 113157 200764 Invocation of 'void com.vmware.vim.binding.vim.StorageResourceManager.recommendDatastores(com.vmware.vim.binding.vim.storageDrs.StoragePlacementSpec,com.vmware.vim.vmomi.core.Future)' for https://Test.test.test.s.com:443/sdk (guid=3de21686-6c7f-40a3-b237-d2967fc43fe3, id=3000613) for opId 'm2fxe2mf-9889226-auto-5vyks-h5:71189500' failed in 600559 ms, error: com.vmware.vim.vmomi.client.exception.ConnectionException: http://localhost:1080/external-vecs/http2/Test.test.test.s.com/443/sdk invocation failed with "java.net.SocketTimeoutException: 600,000 milliseconds timeout on connection http-outgoing-28382 [ACTIVE]"
[YYYY-MM-DDTHH:MM:SS] [WARN ] http-nio-5090-exec-1525 Request processing for opId 'm2fxe2mf-9889226-auto' to URL /ui/mutation/validate/urn%3Avmomi%3AVirtualMachine%3Avm-3018060%3A3de21686-6c7f-40a3-b237-d2967fc43fe3 took too long: 600578 ms
YYYY-MM-DDTHH:MM:SS info vpxd[39097] [Originator@6876 sub=vpxLro opID=m2fxe2mf-9889226-auto-5vyks-h5:71189500-e9] [VpxLRO] -- BEGIN lro-1214853025 -- StorageResourceManager -- vim.StorageResourceManager.recommendDatastores -- 521db0c8-ac86-f3af-b578-26a0a8930554(52ad7d6a-65fa-9f38-13a2-2589464a02a9)
YYYY-MM-DDTHH:MM:SS error vpxd[14140] [Originator@6876 sub=vmomi.soapStub[392875] opID=m2fxe2mf-9889226-auto-5vyks-h5:71189500-e9] Initial service state request failed, disabling pings; /invsvc/vmomi/sdk, , <TCP '127.0.0.1 : 10080'>>>, HTTP Status:400 'Bad Request'
YYYY-MM-DDTHH:MM:SS info vpxd[14066] [Originator@6876 sub=vpxLro opID=sps-Main-328281-438-260643-68] [VpxLRO] -- BEGIN session[5265c884-c1ae-cc2c-2127-8a2431a0fc5a]5295ae6f-c3a1-4a97-566b-dbeae5883cc6 -- CatalogSyncManager -- vim.vslm.vcenter.CatalogSyncManager.queryCatalogChange -- 5265c884-c1ae-cc2c-2127-8a2431a0fc5a(524bf87c-40ea-11db-1a07-d318762e14a7)
YYYY-MM-DDTHH:MM:SS error vpxd[14066] [Originator@6876 sub=HostPicker opID=sps-Main-328281-438-260643-68] Couldn't find any candidate hosts for the provided urls
YYYY-MM-DDTHH:MM:SS warning vpxd[14066] [Originator@6876 sub=Vmomi opID=sps-Main-328281-438-260643-68] VMOMI activation LRO failed; <<5265c884-c1ae-cc2c-2127-8a2431a0fc5a, <TCP '127.0.0.1 : 8085'>, <TCP '127.0.0.1 : 38242'>>, CatalogSyncManager, vim.vslm.vcenter.CatalogSyncManager.queryCatalogChange>, N3Vim5Fault21InaccessibleDatastore9ExceptionE(Fault cause: vim.fault.InaccessibleDatastore
--> )
--> [context]zKq7AVECAQAAAAZLcQEWdnB4ZAAAAto3bGlidm1hY29yZS5zbwAAmXksABdtLQAf6jIBgfBxdnB4ZAABh1B0AUdKlQEGkZWBx6YaAYEBVhoBgv2fPgFsaWJ2aW0tdHlwZXMuc28AgfoBagGB9QRpAYE2BmkBgWQVaQGBx0JoAYGS7GgBAOdJIwB1nyMAwGU3A4d/AGxpYnB0aHJlYWQuc28uMAAEvzYPbGliYy5zby42AA==[/context]
YYYY-MM-DDTHH:MM:SS info vpxd[14066] [Originator@6876 sub=vpxLro opID=sps-Main-328281-438-260643-68] [VpxLRO] -- FINISH session[5265c884-c1ae-cc2c-2127-8a2431a0fc5a]5295ae6f-c3a1-4a97-566b-dbeae5883cc6
YYYY-MM-DDTHH:MM:SS info vpxd[14066] [Originator@6876 sub=Default opID=sps-Main-328281-438-260643-68] [VpxLRO] -- ERROR session[5265c884-c1ae-cc2c-2127-8a2431a0fc5a]5295ae6f-c3a1-4a97-566b-dbeae5883cc6 -- CatalogSyncManager -- vim.vslm.vcenter.CatalogSyncManager.queryCatalogChange: vim.fault.InaccessibleDatastore:
--> Result:
--> (vim.fault.InaccessibleDatastore) {
--> faultCause = (vmodl.MethodFault) null,
--> faultMessage = ,
--> datastore = 'vim.Datastore:3de21686-6c7f-40a3-b237-d2967fc43fe3:datastore-4770264',
--> name = "Datastore-Name-14",
--> detail = "notAccessible"
--> msg = ""
--> }
--> Args:
-->
--> Arg catalogChangeSpec:
--> (vim.vslm.CatalogChangeSpec) {
--> datastore = 'vim.Datastore:datastore-4770264',
--> startVClockTime = (vim.vslm.VClockInfo) {
--> vClockTime = 0
--> },
--> fullSync = false
--> }
YYYY-MM-DDTHH:MM:SS [pool-28-thread-5] INFO opId=sps-Main-328281-438 com.vmware.vim.storage.common.util.OperationIdUtil - OperationID present in invoker thread, adding suffix and re-using it sps-Main-328281-438-260647
YYYY-MM-DDTHH:MM:SS [pool-2-thread-15] INFO opId= com.vmware.vim.storage.common.task.CustomThreadPoolExecutor - [VLSI-client] Active thread count is: 1, Core Pool size is: 20, Queue size: 0, Time spent waiting in queue: 0 millis
YYYY-MM-DDTHH:MM:SS [pool-2-thread-15] INFO opId= com.vmware.vim.storage.common.task.CustomThreadPoolExecutor - [VLSI-client] Request took 4 millis to execute.
YYYY-MM-DDTHH:MM:SS [jaeger.RemoteReporter-QueueProcessor] WARN opId=sps-Main-328281-438 io.jaegertracing.internal.reporters.RemoteReporter - FlushCommand execution failed! Repeated errors of this command will not be logged.
io.jaegertracing.internal.exceptions.SenderException: Failed to flush spans.
... 3 more
Caused by: org.apache.thrift.transport.TTransportException: Cannot flush closed transport
at io.jaegertracing.thrift.internal.reporters.protocols.ThriftUdpTransport.flush(ThriftUdpTransport.java:151) ~[jaeger-thrift-1.6.0.jar:1.6.0]
at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:73) ~[libthrift-0.14.1.jar:0.14.1]
at org.apache.thrift.TServiceClient.sendBaseOneway(TServiceClient.java:66) ~[libthrift-0.14.1.jar:0.14.1]
at io.jaegertracing.agent.thrift.Agent$Client.send_emitBatch(Agent.java:70) ~[jaeger-thrift-1.6.0.jar:1.6.0]
at io.jaegertracing.agent.thrift.Agent$Client.emitBatch(Agent.java:63) ~[jaeger-thrift-1.6.0.jar:1.6.0]
at io.jaegertracing.thrift.internal.senders.UdpSender.send(UdpSender.java:84) ~[jaeger-thrift-1.6.0.jar:1.6.0]
at io.jaegertracing.thrift.internal.senders.ThriftSender.flush(ThriftSender.java:114) ~[jaeger-thrift-1.6.0.jar:1.6.0]
... 3 more
Caused by: java.net.PortUnreachableException: ICMP Port Unreachable
at java.net.PlainDatagramSocketImpl.send(Native Method) ~[?:1.8.0_402]
at java.net.DatagramSocket.send(DatagramSocket.java:693) ~[?:1.8.0_402]
at io.jaegertracing.thrift.internal.reporters.protocols.ThriftUdpTransport.flush(ThriftUdpTransport.java:149) ~[jaeger-thrift-1.6.0.jar:1.6.0]
at org.apache.thrift.TServiceClient.sendBase(TServiceClient.java:73) ~[libthrift-0.14.1.jar:0.14.1]
at org.apache.thrift.TServiceClient.sendBaseOneway(TServiceClient.java:66) ~[libthrift-0.14.1.jar:0.14.1]
at io.jaegertracing.agent.thrift.Agent$Client.send_emitBatch(Agent.java:70) ~[jaeger-thrift-1.6.0.jar:1.6.0]
at io.jaegertracing.agent.thrift.Agent$Client.emitBatch(Agent.java:63) ~[jaeger-thrift-1.6.0.jar:1.6.0]
at io.jaegertracing.thrift.internal.senders.UdpSender.send(UdpSender.java:84) ~[jaeger-thrift-1.6.0.jar:1.6.0]
at io.jaegertracing.thrift.internal.senders.ThriftSender.flush(ThriftSender.java:114) ~[jaeger-thrift-1.6.0.jar:1.6.0]
This issue occurs when Storage DRS cannot place new virtual machines due to one or more of the following reasons:
These conditions stop VM deployment at the cluster level.
To resolve the issue, please follow below:
These actions address both storage capacity constraints and DRS rule conflicts that can prevent VM creation.