Cloning, reconfiguring or deleting VMs getting stuck.
search cancel

Cloning, reconfiguring or deleting VMs getting stuck.

book

Article ID: 301471

calendar_today

Updated On:

Products

VMware vCenter Server 7.0

Issue/Introduction

  • When performing bulk operations such as cloning, reconfiguring, modifying, or deleting virtual machines (VMs) concurrently, the tasks may get stuck or take significantly longer than expected.

  • Stuck tasks are visible in the vCenter Server (VCSA) UI and can appear stuck in various stages, including migrating, powering on, or reconfiguring the VMs.

  • In some cases, the vCenter UI may become unresponsive during this process.


  • The vpxd logs show delays in processing tasks. For example:

    From /var/log/vmware/vpxd.log
    YYYY-MM-YYThh:mm:SS info vpxd[23891] [Originator@6876 sub=VmProv opID=5XXXX7-01] Created VM vim.VirtualMachine:vm-XXXXX (ds:///vmfs/volumes/vsan:52############4-8##########4/9####4-3##2-6##f-00ad-1#####590/test1.vmx) on host [vim.HostSystem:host-#####,<FQDN ESXI host>] (FQDN ESXI host)
  • An Hour Later, Reconfiguration Started:
    YYYY-MM-YYThh:mm:SS info vpxd[23907] [Originator@6876 sub=vpxLro opID=39####7de] [VpxLRO] -- BEGIN task-124050 -- vm-XXXX-- vim.VirtualMachine.reconfigure -- 52XXXb-836c-4XXa-cXX5-5XXXXXXa(5XXX9-f3f9-1XX1-47c2-85XXXXXX25f)


Environment

VMware vCenter server 7.x

Cause

This issue is caused by problems with the HTTP/2 protocol used by Envoy to communicate with the vpxd service. While vpxd acknowledges the requests from Envoy, it fails to process them due to HTTP/2 handling issues, resulting in tasks getting stuck or delayed.

Resolution

This issue is resolved in vCenter Server 7.0 Update 3i (build number 20845200) and vCenter server vCenter Server 8.0a (build number 20920323) onwards. To download the vCenter server ISO follow the link Download vCenter Server patches and ISOs for latest releases

 

Workaround: -

Note: Before proceeding, take a powered-off snapshot of the VCSA VM. If vCenter is deployed in Enhanced Linked Mode (ELM), take powered-off snapshots of all vCenter servers in the environment.

  1. Log in to the vCenter Server Appliance (VCSA) via SSH.

  2. Back up the existing configuration file:

     
    cp /etc/vmware-rhttpproxy/endpoints.conf.d/vpxd-rhttpproxy-endpoint.conf /storage/core

  3. Modify the configuration

    vi /etc/vmware-rhttpproxy/endpoints.conf.d/vpxd-rhttpproxy-endpoint.conf


    Change the following lines:
    From:

     
    /sdk http2:local 8085 redirect allow
    /sdk-vpxd http2:local 8085 reject allow

    To:

    /sdk local 8085 redirect allow
    /sdk-vpxd local 8085 reject allow

  4. Restart the rhttpproxy service to apply the changes:

     
    service-control --restart rhttpproxy

 



Additional Information

Powered by Wolken Software