VCF Workload Domain Deletion Fails with "DELETE_DOMAIN operation validation failed"

Products

• VMware SDDC Manager
• VMware Cloud Foundation

Issue/Introduction

• When attempting to remove a Workload Domain (WLD) within VMware Cloud Foundation, the DELETE_DOMAIN operation fails during the initial validation phase. The SDDC Manager UI show the task as failed, preventing the the removal of the domain from the inventory.

  DELETE_DOMAIN operation validation failed due to: Failed to run guardrail validations of type DELETE_DOMAIN on resource
  

• Log files (/var/log/vmware/vcf/domainmanager/domainmanager.log) report:

  ERROR [vcf_dm, ##############, 0e51] [e.v.v.a. s.AssessmentLambdaServiceClient, dm-exec-22] Encountered exception while executing guardrail validations
  java. lang. IllegalStateException: Failed to evaluate lambda class LambdaInputRest |check8etType: DAY_N_GUARDRAILSdomainIda: [######-####-####-###-########]
  targets: class CheckTargetRest {type: DELETE_DOMAIN
  resourceId: ######-####-####-###-########
  forceRefresh: true
  at com.vmvare. vof.assesament.sdk.LambdaService.runLambda (LambdaService. java: 102)
  at com. vmware. vef.anmemament.ndk. AaseanmentLambdaServiceClient. runAssesamentValidations (AasennmentLambdaServiceClient. java: 121)
  at com.vmware.evo.sddc.common.services.actions.DayNGuardrailsValidationAction.execute (DayNGuardrailsValidat ionAction. java: 113)
  at com.vmware.evo.sddc.common.services.actions.DayNGuardrailaValidationAction.execute (DayNGuardrailsValidationAction.java: 41)
  at com.vmware.evo.sddc.orchestrator.platform.action. FamActionstate. invoke (FamActionState.java: 66)
  
  
  ERROR (vcf_dm, #############, 0e51) [c.v.c.m.c.s.a.DayNGuardrail=ValidationAction,dm-exec-22] Failed to run DELETE DOMAIN validations due to error: Failed to run guardrail validations of type DELETE_DOMAIN on resource ######-####-####-###-########
  ERROR [vef dm,#############, 0e51] [e.v.e.s.o.model.error. ErrorFactory, dm-exec-22] [#####] DAYN GUARDRAILS VALIDATION FAILED DELETE DOMAIN operation validation failed due to! ailed to run quardrail validations of type DELETE DOMAIN on resource ######-####-####-###-########
  com.vmware.evo.sdde.orchestrator.exceptions. OrchTaskException: DELETE DOMAIN operation validation failed due to: Failed to run guardrail validations of type DELETE DOMAIN on resource######-####-####-###-########
  at com. vmvare.evo.sddo. common,services. actiona. DayNGuardrailaValidationAction.execute (DayNGuardrailsValidationAction. java: 130)
  at com.vmware. evo.sdde.common.services.actions. DayNGuardrailsValidationAetion.execute (DayNGuardrailsValidat ionAction. java : 41)
  
  ERROR [vcf_dm,#############,cbcl] [c.v.v.c.service. ResourceCacheService, dm-exec-14] Error happened during isConfigRealizedCheck on resource ######-####-####-###-########
  of drift######-####-####-###-########. Will re-try in the next refresh cycle.
  com. vmware. vef.configreconciler. common. error. ConfigReconcilerCommonException: Failed to check if configuration associated with drift######-####-####-###-######## is already realized on resource No route to host duet (2). The system will automatically retry this check in the next cache refresh cycle.
  at com. vmware. vcf. configreconciler. config. checks. ComputeManagerSettingsDriftCheck. isConfigurationRealized(ComputeManagerSettingaDriftCheck. java: 191)
  at com. vmware. vcf. configreconciler. service. ResourceCacheService.performIsConfigRealizedCheck (ResourceCacheService. java: 978)
  at com. vmware. vcf.configreconciler.service.ResourceCacheService. refreshFailedConfigRealizedCheck (ResourceCacheService. java: 1017)
  at com. vmware. vcf.configreconciler.service.

Environment

VMware Cloud Foundation (VCF) 9.0

Cause

The workflow was blocked by an inconsistent resource state in the SDDC Manager database and a subsequent communication failure (No route to host) during the Assessment Lambda service (responsible for Guardrail checks) guardrail checks.

Because the guardrail check could not complete, the deletion workflow was blocked.

Resolution

Contact Broadcom Support for assistance