"Failed to update and re-apply vSAN storage policy VsanDefaultProfile" error when attempting to stretch a vSAN cluster in VMware Cloud Foundation
search cancel

"Failed to update and re-apply vSAN storage policy VsanDefaultProfile" error when attempting to stretch a vSAN cluster in VMware Cloud Foundation

book

Article ID: 371032

calendar_today

Updated On:

Products

VMware SDDC Manager VMware vSAN 8.x

Issue/Introduction

This issue can occur while stretching a vSAN Cluster in VMware Cloud Foundation

Symptoms:

 

  • The workflow will fail and show the following error:

Description Update vSAN Storage Profile
Progress Messages Failed to update and re-apply vSAN storage policy VsanDefaultProfile
Failed to update and re-apply vSAN storage policy VsanDefaultProfile
Failed to update and re-apply vSAN storage policy VsanDefaultProfile
Failed to update and re-apply vSAN storage policy VsanDefaultProfile
Error

Message: Failed to update and re-apply vSAN storage policy VsanDefaultProfile
Remediation Message:

Reference Token: XXXXXX

Cause: Exception [Task: (MOR:task-792) (Name:reconfigure) Entity: (MOR:vm-40) (Name:vm.domain.com) is failed] while waiting for reconfiguring task to end task id : task-792 Task: (MOR:task-792) (Name:reconfigure) Entity: (MOR:vm-40) (Name:vm.domain.com) is failed Invalid virtual machine configuration.

 

  • SDDC manager logs contains errors similar to the excerpt below:

    ERROR [vcf_dm,65a8eb803d1ab58bcee1b51d8b11e5a3,154e] [c.v.e.s.c.c.v.pbm.PbmManagerBase,dm-exec-18]  Exception [Task: (MOR:task-1448) (Name:reconfigure) Entity: (MOR:vm-1027) (Name:witness-vm.domain.com) is failed] while waiting for  reconfiguring task to end  task id : task-1448
    ERROR [vcf_dm,65a8eb803d1ab58bcee1b51d8b11e5a3,154e] [c.v.e.s.o.model.error.ErrorFactory,dm-exec-18]  [2TT1M4] VSAN_DEFAULT_STORAGE_POLICY_UPDATION_FAILED Failed to update and re-apply vSAN storage policy VsanDefaultProfile
    com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: Failed to update and re-apply vSAN storage policy VsanDefaultProfile
            at com.vmware.evo.sddc.stretchclustermanager.tasks.UpdateVsanStorageProfileAction.execute(UpdateVsanStorageProfileAction.java:86)
            at com.vmware.evo.sddc.stretchclustermanager.tasks.UpdateVsanStorageProfileAction.execute(UpdateVsanStorageProfileAction.java:32)
            at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionState.invoke(FsmActionState.java:62)
            at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionPlugin.invoke(FsmActionPlugin.java:159)
            at com.vmware.evo.sddc.orchestrator.platform.action.FsmActionPlugin.invoke(FsmActionPlugin.java:144)
            at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.invokeMethod(ProcessingTaskSubscriber.java:400)
            at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.processTask(ProcessingTaskSubscriber.java:520)
            at com.vmware.evo.sddc.orchestrator.core.ProcessingTaskSubscriber.accept(ProcessingTaskSubscriber.java:124)
            at jdk.internal.reflect.GeneratedMethodAccessor900.invoke(Unknown Source)
            at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            at java.base/java.lang.reflect.Method.invoke(Method.java:568)
            at com.google.common.eventbus.Subscriber.invokeSubscriberMethod(Subscriber.java:85)
            at com.google.common.eventbus.Subscriber.lambda$dispatchEvent$0(Subscriber.java:71)
            at com.vmware.vcf.common.tracing.TraceRunnable.run(TraceRunnable.java:59)
            at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
            at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
            at java.base/java.lang.Thread.run(Thread.java:840)
    Caused by: com.vmware.evo.sddc.common.client.vmware.vsphere.VsphereOperationException: Exception [Task: (MOR:task-1448) (Name:reconfigure) Entity: (MOR:vm-1027) (Name:witness-vm.domain.com) is failed] while waiting for  reconfiguring task to end  task id : task-1448
            at com.vmware.evo.sddc.common.client.vmware.pbm.PbmManagerBase.checkPolicyReconfigOutcomesForFault(PbmManagerBase.java:1021)
            at com.vmware.evo.sddc.common.client.vmware.pbm.PbmManagerBase.reapplyStoragePolicy(PbmManagerBase.java:988)
            at com.vmware.evo.sddc.stretchclustermanager.tasks.UpdateVsanStorageProfileAction.reapplyStoragePolicy(UpdateVsanStorageProfileAction.java:148)
            at com.vmware.evo.sddc.stretchclustermanager.tasks.UpdateVsanStorageProfileAction.execute(UpdateVsanStorageProfileAction.java:73)
            ... 16 common frames omitted



  • vCenter logs contains errors similar to the excerpt below:


    /var/log/vmware/vpxd/vpxd.log

    error vpxd[20703] [Originator@6876 sub=Default opID=973e388c-e438-4173-9f11-823014087266-389587-213-91-01] [VpxLRO] -- ERROR lro-1185202 -- 52adb303-beec-b2a4-07e8-9875b0e291fb(52a29f9f-8d01-cc72-843d-92052c2b9c8b) -- vm-34 -- vim.fault.InvalidVmConfig
    --> Result:
    --> (vim.fault.InvalidVmConfig) {
    --> faultCause = (vmodl.MethodFault) null,
    --> faultMessage = (vmodl.LocalizableMessage) [
    --> (vmodl.LocalizableMessage) {
    --> key = "vob.vsan.clomd.needMoreFaultDomains2",
    --> arg = (vmodl.KeyAnyValue) [
    --> (vmodl.KeyAnyValue) {
    --> key = "1",
    --> value = "2"
    --> },
    --> (vmodl.KeyAnyValue) {
    --> key = "2",
    --> value = "1"
    --> }
    --> ],
    --> message = "There are currently 2 usable fault domains. The operation requires 1 more usable fault domains."
    --> }
    --> ],
    --> property = "config.vmProfile"
    --> msg = "Invalid virtual machine configuration."
    --> }
    --> ]

    /var/log/vmware/vpxd/sps.log

    [pool-27-thread-4] ERROR opId=sps-Main-45626-621 com.vmware.vim.sms.provider.vasa.alarm.AlarmDispatcher - Error occurred while polling alarms for provider: https://witness-vm.domain.com:9080/version.xml
    com.vmware.vim.sms.fault.VasaServiceException: org.apache.axis2.AxisFault: No route to host (Host unreachable)

 

Environment

VMware Cloud foundation 5.x

VMware vSAN 8.x

Cause

The error is indicating a network connectivity issue between the ESXi and the witness VM outside of vSAN.

Resolution

This is an environmental issue. Investigate for blocked traffic between the ESXi and the Witness VM.

Port connectivity requirements for vSAN Witness:

https://ports.esp.vmware.com/home/vSphere

(Filter "Destination: vSAN Witness")

Additional Information

Powered by Wolken Software