Workload domain deployment fails when a VM MANAGEMENT port group gets created on the wrong vSphere Distributed Switch (VDS)

Products

• VMware SDDC Manager

Issue/Introduction

If your bring-up spec contains multiple vSphere Distributed Switches, the VM MANAGEMENT port group may get created on the VDS that contains the MANAGEMENT port group, even if you specified a different VDS for VM MANAGEMENT and MANAGEMENT.

The WLD creation will fail at step "Generate Workload Domain Runtime Data Model" and you will see errors similar to this:

Found invalid number of management portgroups in the default cluster of the management workload domain. Expected 1 portgroups but found 5.
com.vmware.evo.sddc.orchestrator.exceptions.OrchTaskException: Found invalid number of management portgroups in the default cluster of the management workload domain. Expected 1 portgroups but found 5.

Environment

VCF 5.1.1

Cause

This is a known issue caused by how the WLD spec is formatted/parsed. 

Resolution

This issue can be prevented by specifying the VDS for the VM MANAGEMENT network before the VDS used for MANAGEMENT in spec.

You can workaround this issue by updating the impacted VDS in the psql database to remove VM_MANAGEMENT port group.

1. Take a snapshot of SDDC manager VM. 
2. ssh to SDDC Manager with 'vcf' user and su to 'root'
3. Identify the vds that should not be tagged with VM_MANAGEMENT and needs to be removed.
   • /usr/pgsql/13/bin/psql -U postgres -h localhost -d platform;
   • platform=# select id,name,port_groups from vds;
     • [{"name":"example-pg-mgmt","transportType":"MANAGEMENT","type":"EPHEMERAL","vlanId":800,"activeUplinks":["uplink1","uplink2"],"id":"111111-1111-1111-1111-111111111111"},
{"name":"example-pg-vmotion","transportType":"VMOTION","type":"EARLY_BINDING","vlanId":801,"activeUplinks":["uplink1","uplink2"],"id":"222222-2222-2222-2222-222222222222"},{"name":"example-pg-vm-mgmt","transportType":"VM_MANAGEMENT",
"type":"EPHEMERAL","vlanId":804,"activeUplinks":["uplink1","uplink2"],"id":"333333-3333-3333-3333-333333333333"}]
4. Update the DB to remove this information
   • platform=# update vds set port_groups ='[{"name":"example-pg-mgmt","transportType":"MANAGEMENT","type":"EPHEMERAL","vlanId":800,"activeUplinks":["uplink1","uplink2"],"id":"111111-1111-1111-1111-111111111111"},
{"name":"example-pg-vmotion","transportType":"VMOTION","type":"EARLY_BINDING","vlanId":801,"activeUplinks":["uplink1","uplink2"],"id":"222222-2222-2222-2222-222222222222"}' WHERE id='444444-4444-4444-4444-444444444444';
5. Retry the failed workflow.

Additional Information

https://docs.vmware.com/en/VMware-Cloud-Foundation/5.1.1/rn/vmware-cloud-foundation-511-release-notes/index.html