Cannot power on vCloud Director VMs with guest customization enabled
search cancel

Cannot power on vCloud Director VMs with guest customization enabled

book

Article ID: 325481

calendar_today

Updated On:

Products

VMware Cloud Director

Issue/Introduction

Symptoms:
When ESXi hosts in Provider cluster are in standby mode, you experience these symptoms:
  • When powering on a virtual machine in vCloud Director which has guest customization enabled, the virtual machine is placed on the active host which is not in Standby Mode but the task to place the CAB file for customization seems to try to go through one of the hosts which are in Standby Mode.
  • In vCloud Director, the ESXi hosts are shown with two System Alerts:
     
    • The ESXi host is unavailable
    • The ESXi is in Standby Mode
       
  • You see error trace similar to:

    2014-08-08 15:45:21,609 | DEBUG | backend-activity-pool-319 | CJob | updateFailedJob(com.vmware.ssdc.util.LMException) with locale=null | vcd=<VCD_UUID>,task=<TASK_UUID> activity=urn:uuid:<TASK_UUID>
    com.vmware.ssdc.util.LMException: Unable to start vApp "<VAPP_NAME>".
    at com.vmware.ssdc.backend.AbstractVAppImpl.deploySubset(AbstractVAppImpl.java:526)
    at com.vmware.ssdc.backend.services.impl.VAppDeploymentManagerImpl.deployVm(VAppDeploymentManagerImpl.java:398)
    at com.vmware.vcloud.vapp.impl.VAppServiceImpl.deployTask(VAppServiceImpl.java:1894)
    at com.vmware.vcloud.vapp.impl.VAppServiceImpl.executeTask(VAppServiceImpl.java:691)
    at com.vmware.vcloud.backendbase.management.system.TaskActivity$ExecutePhase$2.doInSecurityContext(TaskActivity.java:427)
    at com.vmware.vcloud.backendbase.management.system.TaskActivity$ExecutePhase$2.doInSecurityContext(TaskActivity.java:421)
    at com.vmware.vcloud.backendbase.management.system.TaskActivity$SecurityContextTemplate.executeForOrgAndUser(TaskActivity.java:674)
    at com.vmware.vcloud.backendbase.management.system.TaskActivity$ExecutePhase.execute(TaskActivity.java:421)
    at com.vmware.vcloud.backendbase.management.system.TaskActivity$ExecutePhase.invoke(TaskActivity.java:354)
    at com.vmware.vcloud.activity.executors.ActivityRunner.run(ActivityRunner.java:109)
    at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
    at java.util.concurrent.FutureTask$Sync.innerRun(Unknown Source)
    at java.util.concurrent.FutureTask.run(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
    at java.lang.Thread.run(Unknown Source)
    Caused by: com.vmware.ssdc.util.LMException: HTTP NFC Error. Cannot deploy package from /opt/vmware/vcloud-director/tmp/<CAB_FILENAME> to '<VM_FILE_PATH>/<CAB_FILENAME>'.
    at com.vmware.vcloud.customization.impl.DeploymentPackage.DeployPackage(DeploymentPackage.java:251)
    at com.vmware.vcloud.customization.impl.CustomizationServiceImpl.pushDeploymentPackage(CustomizationServiceImpl.java:196)
    at com.vmware.vcloud.customization.impl.VmGuestCustomizationActivity$PushDeploymentPackagePhase.invoke(VmGuestCustomizationActivity.java:80)
    ... 7 more
    Caused by: com.vmware.vcloud.fabric.storage.copy.CompleteFailureException
    at com.vmware.vcloud.fabric.storage.copy.impl.HttpFileCopyService.handleActivityFuture(HttpFileCopyService.java:203)
    at com.vmware.vcloud.fabric.storage.copy.impl.HttpFileCopyService.copyToDatastore(HttpFileCopyService.java:123)
    at sun.reflect.GeneratedMethodAccessor6227.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
    at java.lang.reflect.Method.invoke(Unknown Source)
    at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:319)
    at org.springframework.osgi.service.importer.support.internal.aop.ServiceInvoker.doInvoke(ServiceInvoker.java:58)
    at org.springframework.osgi.service.importer.support.internal.aop.ServiceInvoker.invoke(ServiceInvoker.java:62)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
    at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.osgi.service.util.internal.aop.ServiceTCCLInterceptor.invokeUnprivileged(ServiceTCCLInterceptor.java:56)
    at org.springframework.osgi.service.util.internal.aop.ServiceTCCLInterceptor.invoke(ServiceTCCLInterceptor.java:39)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.osgi.service.importer.support.LocalBundleContextAdvice.invoke(LocalBundleContextAdvice.java:59)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.support.DelegatingIntroductionInterceptor.doProceed(DelegatingIntroductionInterceptor.java:131)
    at org.springframework.aop.support.DelegatingIntroductionInterceptor.invoke(DelegatingIntroductionInterceptor.java:119)
    at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
    at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:202)
    at com.sun.proxy.$Proxy736.copyToDatastore(Unknown Source)
    at com.vmware.vcloud.customization.impl.DeploymentPackage.DeployPackage(DeploymentPackage.java:248)
    ... 9 more
    Caused by: com.vmware.ssdc.util.LMException: java.net.NoRouteToHostException: No route to host


    Note: The preceding log excerpts are only examples. Date, time, and environmental variables may vary depending on your environment.  


Resolution

To resolve the issue, manually disable the ESXi hosts which are in Standby mode from vCloud Director. vCloud Director now no longer tries to use them to place the CAB file and the virtual machine powers on and customize.