VM deployment from template or cloning tasks stall or take an unusually long time in vCenter 8.0 U2

Products

• VMware vCenter Server
• VMware vCenter Server 8.0

Issue/Introduction

• VMware vCenter Server 8.0 U2 environments may experience severe performance degradation or hangs during OVF deployments and VM cloning. Tasks typically stall at 95% and can take upwards of 80 minutes to complete. 

• Some of the VMs could easily take 50 to 80 minutes for its creation even if the disk size is only a few GBs.
• The following errors are captured in the vCenter vpxd.log (/var/log/vmware/vpxd/vpxd.log): 

[timestamp] verbose vpxd[05975] [Originator@6876 sub=Http2ServerSession-1 opID=1b5##ab1] Releasing stream; {<io_obj p:0x00007fe7e4063310, h:82, <TCP '127.0.0.1 : 8085'>, <TCP '127.0.0.1 : 58446'>>, id: 48879289, state(in/out): 3/3}, active: 7
[timestamp] verbose vpxd[05975] [Originator@6876 sub=Http2ServerSession-1 opID=1b5##ab1] OnStreamClose on stream 48879289 with errcode 0
[timestamp] error vpxd[06477] [Originator@6876 sub=IO.Http opID=lug##cgf-271##10-auto-1m92s-h5:70##2197-fd-01] User agent failed to send request; (null), N7Vmacore16TimeoutExceptionE(Operation timed out)
--> [context]zKq7AVECAQAAAIUcWQE1dnB4ZAAAxbVTbGlidm1hY29yZS5zbwAAUglDAIwxRADMSksAa5dIAbxNIGxpYnZtb21pLnNvAAGdxx8BfTYaAoRUAmxpYnZzbS10eXBlcy5zbwCD4b5FAnZweGQAg3PDRQKD9sNFAoOExUUCAX02GgHNKhqEPtAmAW##eXBlcy5zbwCDa6+7AYNfsrsBg0LwMQGDEl0iAoNRYCICg1tZHgKD9lkeAoPTTCUCg+lNJQKDQ/4kAoN9CGICg88YYgKD+S9hAoNSi2ECg6eVGgKDIZcaAoMijhoCg7mOGgKDIo4aAoO3kRoCg2ySGgKDWpsaAoPxNdcBg4s21wGEfgIyAYO5d2MCAdXDG4MgokMCg30IYgKDzxhiAoP5L2ECg5QCYgIA5ss3APkkOACTwFEFro4AbGlicHRocmVhZC5zby###WJjLnNvLjYA[/context]
[timestamp] error vpxd[06437] [Originator@6876 sub=OvfConsumers opID=lug##cgf-271##10-auto-1m92s-h5:70##2197-fd-01] Failed to authenticate with VSM; stubAdapter: <<cs p:000055bf61d18440, TCP:localhost:1080>, /vsm/sdk>, N7Vmacore16TimeoutExceptionE(Operation timed out)

• The following errors are captured in the vCenter vsm.log (/var/log/vmware/vsm/vsm.log)

[timestamp] ERROR [pool-5-thread-1] VsmActivationValidator.java 267 - Failed to validate user: only vpxd-svc-acct requests allowed and not

Environment

VMware vCenter 8.0 U2

Cause

This is a known issue in vCenter 8.0 U2. OVF deployments and VM cloning operations are delayed because the vService Manager (VSM) fails to generate SAML tokens efficiently.

There are two primary factors contributing to this behavior:

• Service Account Expiration: The VSM service account password expires every 90 days. When this occurs, the vCenter code attempts to reset the password automatically. In this version, the process fails to complete, causing the task to hang indefinitely.

• FQDN Case Sensitivity: The internal Envoy proxy treats Fully Qualified Domain Names (FQDNs) as case-sensitive. If the vCenter FQDN is configured with uppercase letters, Envoy may fail to recognize the route, preventing VSM from connecting to vCenter services.

Resolution

This is a known issue with vCenter 8.0U2 and is resolved in patch 8.0U3. 

As a workaround, restart the VSM service to recreate the service account. 

1. SSH to vCenter via root
2. Restart the VSM service:
   service-control --restart vsm

OR

1. Login to the vCenter Appliance Management Interface (VAMI) as root: https://<vCenter_FQDN_or_IP_ADDRESS>:5480
2. Go to Services
3. Select the VMware vService Manager and click Restart

Note: Every time the VSM service is restarted, a new service account and password is created.