SDDC Manager Stretch cluster expansion is failing at - Validate Expand Stretch Cluster ESXi Host(s) Network
search cancel

SDDC Manager Stretch cluster expansion is failing at - Validate Expand Stretch Cluster ESXi Host(s) Network

book

Article ID: 409399

calendar_today

Updated On:

Products

VMware SDDC Manager VMware Cloud Foundation

Issue/Introduction

  • SDDC Manager Stretch cluster expansion is failing at
    Validate Expand Stretch Cluster ESXi Host(s) Network

     

  • Error in SDDC Manager UI
    Description				Validate Expand Stretch Cluster ESXi Host(s) Network
    
    Progress Messages		Failed to validate host's network.
    
    Error					Message: Failed to validate host's network.
    
    						Remediation Message: Please make sure host management VLAN id and networkPoolId matches the specified fault domain.
    
    						Reference Token: 4####1
    
    						Cause: Failed to validate host's network.

     

  • Error in SDDC Manager /var/log/VMware/vcf/domainmanager.log
    ERROR [vcf_dm,68b###################081,f82e] [c.v.e.s.s.t.ValidateExpandStretchHostNetworkAction,dm-exec-16]  Host az2-esx05.example.com NetworkPoolId 2104####-####-####-####-########80c4 is not matching availability zones NetworkPoolId a105####-####-####-####-########4777a

     

    ERROR [vcf_dm,68b###################081,f82e] [c.v.e.s.s.t.ValidateExpandStretchHostNetworkAction,dm-exec-16]  Validating hosts NetworkPoolId and Management VlanId failed Failed to validate hosts network.
    ERROR [vcf_dm,68b###################081,f82e] [c.v.e.s.o.model.error.ErrorFactory,dm-exec-16]  [4DGND1] HOST_NETWORK_VALIDATION_FAILED Failed to validate hosts network.
    com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: Failed to validate hosts network.
            at com.vmware.evo.sddc.stretchclustermanager.tasks.ValidateExpandStretchHostNetworkAction.execute(ValidateExpandStretchHostNetworkAction.java:111)
            at com.vmware.evo.sddc.stretchclustermanager.tasks.ValidateExpandStretchHostNetworkAction.execute(ValidateExpandStretchHostNetworkAction.java:27)
           
    Caused by: com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: Failed to validate hosts network.
            at com.vmware.evo.sddc.stretchclustermanager.tasks.ValidateExpandStretchHostNetworkAction.validateNetworkPool(ValidateExpandStretchHostNetworkAction.java:162)
            at com.vmware.evo.sddc.stretchclustermanager.tasks.ValidateExpandStretchHostNetworkAction.execute(ValidateExpandStretchHostNetworkAction.java:104)
            ... 16 common frames omitted

     

  • Find the Preferred FaultDomain from SDDC Manager /var/log/VMware/vcf/domainmanager.log
    DEBUG [vcf_dm,68###############b1,e45a] [c.v.e.s.c.c.v.vsan.VsanManagerBase,dm-exec-2]  Preferred FaultDomain Name example_secondary-az-faultdomain
    DEBUG [vcf_dm,68###############b1,e45a] [c.v.e.s.c.c.v.vsan.VsanManagerBase,dm-exec-2]  Secondary FaultDomain Host Names [az2-esx01.example.com, az2-esx02.example.com, az2-esx03.example.com, az2-esx04.example.com]

Environment

VMware Cloud Foundation

Cause

Mismatched host alignment as per FaultDomain in the ClusterExpansion Spec file

Resolution

To resolve the issue follow either of the below methods

 

Method 1

Change the Preferred FaultDomain to example_primary-az-faultdomain

  1. Login to vSphere UI
  2. Browse to Cluster > Configure > vSAN > Fault Domain
  3. Click on the 3 dots on the example_primary-az-faultdomain and set as Preferred

 

Method 2

Align the hosts as per the azName in the clusterExpansionSpec

Sample clusterExpansionSpec

Before

{
   "clusterExpansionSpec" : {
      "deployWithoutLicenseKeys" : true,
      "hostSpecs" : [
         {
            "azName" : "example_primary-az-faultdomain",
            "hostname" : "az2-esx05.example.com",
            "hostNetworkSpec" : {
               "vmNics" : [
                  {
                     "id" : "vmnic0",
                     "uplink" : "uplink1",
                     "vdsName" : "example-clu01-vds01-mgt"
                  },
                  {
                     "id" : "vmnic2",
                     "uplink" : "uplink2",
                     "vdsName" : "example-clu01-vds01-mgt"
                  },
                  {
                     "id" : "vmnic1",
                     "uplink" : "uplink1",
                     "vdsName" : "example-clu01-vds01-nsx"
                  },
                  {
                     "id" : "vmnic3",
                     "uplink" : "uplink2",
                     "vdsName" : "example-clu01-vds01-nsx"
                  }
               ]
            },
            "id" : "09b2####-####-####-####-########80dc",
            "ipAddress" : "192.0.2.5"
         },
         {
            "azName" : "example_secondary-az-faultdomain",
            "hostname" : "az1-esx05.example.com",
            "hostNetworkSpec" : {
               "vmNics" : [
                  {
                     "id" : "vmnic0",
                     "uplink" : "uplink1",
                     "vdsName" : "example-clu01-vds01-mgt"
                  },
                  {
                     "id" : "vmnic2",
                     "uplink" : "uplink2",
                     "vdsName" : "example-clu01-vds01-mgt"
                  },
                  {
                     "id" : "vmnic1",
                     "uplink" : "uplink1",
                     "vdsName" : "example-clu01-vds01-nsx"
                  },
                  {
                     "id" : "vmnic3",
                     "uplink" : "uplink2",
                     "vdsName" : "example-clu01-vds01-nsx"
                  }
               ]
            },
            "id" : "0a05####-####-####-####-########fbfc",
            "ipAddress" : "192.0.3.5"
         }
      ]
   }
}

 

After

  • host az1-esx05.example.com moved from example_secondary-az-faultdomain to example_primary-az-faultdomain and host az2-esx05.example.com moved from example_primary-az-faultdomain to example_secondary-az-faultdomain
{
   "clusterExpansionSpec" : {
      "deployWithoutLicenseKeys" : true,
      "hostSpecs" : [
         {
            "azName" : "example_primary-az-faultdomain",
            "hostname" : "az1-esx05.example.com",
            "hostNetworkSpec" : {
               "vmNics" : [
                  {
                     "id" : "vmnic0",
                     "uplink" : "uplink1",
                     "vdsName" : "example-clu01-vds01-mgt"
                  },
                  {
                     "id" : "vmnic2",
                     "uplink" : "uplink2",
                     "vdsName" : "example-clu01-vds01-mgt"
                  },
                  {
                     "id" : "vmnic1",
                     "uplink" : "uplink1",
                     "vdsName" : "example-clu01-vds01-nsx"
                  },
                  {
                     "id" : "vmnic3",
                     "uplink" : "uplink2",
                     "vdsName" : "example-clu01-vds01-nsx"
                  }
               ]
            },
            "id" : "09b2####-####-####-####-########80dc",
            "ipAddress" : "192.0.2.5"
         },
         {
            "azName" : "example_secondary-az-faultdomain",
            "hostname" : "az2-esx05.example.com",
            "hostNetworkSpec" : {
               "vmNics" : [
                  {
                     "id" : "vmnic0",
                     "uplink" : "uplink1",
                     "vdsName" : "example-clu01-vds01-mgt"
                  },
                  {
                     "id" : "vmnic2",
                     "uplink" : "uplink2",
                     "vdsName" : "example-clu01-vds01-mgt"
                  },
                  {
                     "id" : "vmnic1",
                     "uplink" : "uplink1",
                     "vdsName" : "example-clu01-vds01-nsx"
                  },
                  {
                     "id" : "vmnic3",
                     "uplink" : "uplink2",
                     "vdsName" : "example-clu01-vds01-nsx"
                  }
               ]
            },
            "id" : "0a05####-####-####-####-########fbfc",
            "ipAddress" : "192.0.3.5"
         }
      ]
   }
}

 

Expand the Stretch cluster - Refer Expand a Stretched Cluster in VMware Cloud Foundation