Disabling Port Group Reconfigure During Cell Site Host Provisioning
Article ID: 325384
Updated On:
Products
VMware
VMware Telco Cloud Automation
Issue/Introduction
Symptoms:
In TCA 2.3, there is an issue which causes a port group reconfigure action to be triggered when cell site hosts are picked up for provisioning in a domain. When this reconfigure action is triggered, if there are any disconnected or unreachable hosts attached to the VDS then the reconfigure fails, thereby failing the provisioning of the hosts that were picked up and they are marked as FAILED.
This issue mainly happens if any of the port groups have vmnics that are marked to be set as unused uplinks.
Error:
In TCA 2.3, there is an issue which causes a port group reconfigure action to be triggered when cell site hosts are picked up for provisioning in a domain. When this reconfigure action is triggered, if there are any disconnected or unreachable hosts attached to the VDS then the reconfigure fails, thereby failing the provisioning of the hosts that were picked up and they are marked as FAILED.
This issue mainly happens if any of the port groups have vmnics that are marked to be set as unused uplinks.
Error:
Failed to reconfigure the vds: hostname reason: (vim.fault.DvsOperationBulkFault) {
dynamicType = <unset>,
dynamicProperty = (vmodl.DynamicProperty) [],
msg = 'Cannot complete a vSphere Distributed Switch operation for one or more host members.',
faultCause = <unset>,
faultMessage = (vmodl.LocalizableMessage) [],
hostFault = (vim.fault.DvsOperationBulkFault.FaultOnHost)
Environment
VMware Telco Cloud Automation 2.3
Resolution
Issue will be resolved in TCA 2.3.0.2 and 3.1.
Workaround:
This issue can be worked around using one of the two procedures listed in this article. Both procedures have been documented below so select the one that is more appropriate for your environment.
Procedure 1:
If the unreachable or disconnected hosts are removed from the relevant VDS containing the port groups that have unused uplinks, then provisioning of new hosts in this cell site group will succeed and any hosts previously failed due to this reason can be provisioned successfully after a resync.
Procedure 2:
The second workaround involves disabling the trigger to the port group reconfigure action i.e., disable functionality to allow changing of VLAN ID, Mac Learning Enablement, Uplink Port Order after the host provisioning has been done for a given VDS and port group combination in a cell site group.
However, because the functionality is being disabled, it means that even if a user changes the configuration in the cell site group and resyncs any existing hosts or adds new hosts, the port group configuration change is not performed. Therefore, any of these types of changes must be done manually in the relevant vCenter.
self, pg_name: str, vlan_id: int, mac_learning_enabled: bool,
uplink_port_order: UplinkPortOrder) -> Optional[vim.dvs.DistributedVirtualPortgroup]:
"""
This method creates the specified port group if it is not present.
If it is already present, then it is modified according to the specified configuration.
@param pg_name: The name of the port group to be created/modified.
@param vlan_id: The value for VLAN. If 0 is specified, then it means no VLAN.
@param mac_learning_enabled: Whether to enabled mac learning or not.
@param uplink_port_order: The grouping and ordering of the uplinks.
@return The process port group
@rtype Optional[vim.dvs.DistributedVirtualPortgroup]
"""
pg = self.get_portgroup(pg_name)
if pg:
# PG already exists, so check if its current configuration is same as desired configuration
existing_pg_spec = self.__compute_comparison_spec_from_pg(pg)
desired_pg_spec = self.__compute_comparison_spec(vlan_id, mac_learning_enabled, uplink_port_order)
# Complex Named Tuples can still be directly compared for equality
if existing_pg_spec == desired_pg_spec:
# Current config and desired config are same, so return existing PG object/instance
return pg
# Current and desired config are different, so reconfigure the PG
new_spec = self.__compute_reconfigure_portgroup_spec(
pg_name, pg.config.configVersion, vlan_id, mac_learning_enabled, uplink_port_order)
self.__reconfigure_portgroup(pg, new_spec)
else:
# PG doesn't exist, so create it
pg_spec = self.create_port_group_spec(pg_name, vlan_id, mac_learning_enabled, uplink_port_order)
self.add_portgroup(pg_spec)
return self.get_portgroup(pg_name)
def create_or_modify_portgroup(
self, pg_name: str, vlan_id: int, mac_learning_enabled: bool,
uplink_port_order: UplinkPortOrder) -> Optional[vim.dvs.DistributedVirtualPortgroup]:
"""
This method creates the specified port group if it is not present.
If it is already present, then it is modified according to the specified configuration.
@param pg_name: The name of the port group to be created/modified.
@param vlan_id: The value for VLAN. If 0 is specified, then it means no VLAN.
@param mac_learning_enabled: Whether to enabled mac learning or not.
@param uplink_port_order: The grouping and ordering of the uplinks.
@return The process port group
@rtype Optional[vim.dvs.DistributedVirtualPortgroup]
"""
pg = self.get_portgroup(pg_name)
if pg:
return pg
# PG already exists, so check if its current configuration is same as desired configuration
existing_pg_spec = self.__compute_comparison_spec_from_pg(pg)
desired_pg_spec = self.__compute_comparison_spec(vlan_id, mac_learning_enabled, uplink_port_order)
# Complex Named Tuples can still be directly compared for equality
if existing_pg_spec == desired_pg_spec:
# Current config and desired config are same, so return existing PG object/instance
return pg
# Current and desired config are different, so reconfigure the PG
new_spec = self.__compute_reconfigure_portgroup_spec(
pg_name, pg.config.configVersion, vlan_id, mac_learning_enabled, uplink_port_order)
self.__reconfigure_portgroup(pg, new_spec)
else:
# PG doesn't exist, so create it
pg_spec = self.create_port_group_spec(pg_name, vlan_id, mac_learning_enabled, uplink_port_order)
self.add_portgroup(pg_spec)
return self.get_portgroup(pg_name)
systemctl restart tcf-manager-deploy
systemctl start tcf-manager
Workaround:
This issue can be worked around using one of the two procedures listed in this article. Both procedures have been documented below so select the one that is more appropriate for your environment.
Procedure 1:
If the unreachable or disconnected hosts are removed from the relevant VDS containing the port groups that have unused uplinks, then provisioning of new hosts in this cell site group will succeed and any hosts previously failed due to this reason can be provisioned successfully after a resync.
Procedure 2:
The second workaround involves disabling the trigger to the port group reconfigure action i.e., disable functionality to allow changing of VLAN ID, Mac Learning Enablement, Uplink Port Order after the host provisioning has been done for a given VDS and port group combination in a cell site group.
However, because the functionality is being disabled, it means that even if a user changes the configuration in the cell site group and resyncs any existing hosts or adds new hosts, the port group configuration change is not performed. Therefore, any of these types of changes must be done manually in the relevant vCenter.
- Ensure that there are no active workflows or workloads running on TCA and in the Infrastructure Automation component.
- Take a snapshot of the TCA virtual machine for backup purposes.
- SSH into the TCA Manager appliance.
- Enter the Infrastructure Automation docker container with the command:
- Make a backup of the file named vcapi.py with the following command:
- Open the file named vcapi.py for editing with VIM and navigate to line number 1135. The cursor should now be inside a function with the name create_or_modify_portgroup and it should look like the following code snippet before modification.
self, pg_name: str, vlan_id: int, mac_learning_enabled: bool,
uplink_port_order: UplinkPortOrder) -> Optional[vim.dvs.DistributedVirtualPortgroup]:
"""
This method creates the specified port group if it is not present.
If it is already present, then it is modified according to the specified configuration.
@param pg_name: The name of the port group to be created/modified.
@param vlan_id: The value for VLAN. If 0 is specified, then it means no VLAN.
@param mac_learning_enabled: Whether to enabled mac learning or not.
@param uplink_port_order: The grouping and ordering of the uplinks.
@return The process port group
@rtype Optional[vim.dvs.DistributedVirtualPortgroup]
"""
pg = self.get_portgroup(pg_name)
if pg:
# PG already exists, so check if its current configuration is same as desired configuration
existing_pg_spec = self.__compute_comparison_spec_from_pg(pg)
desired_pg_spec = self.__compute_comparison_spec(vlan_id, mac_learning_enabled, uplink_port_order)
# Complex Named Tuples can still be directly compared for equality
if existing_pg_spec == desired_pg_spec:
# Current config and desired config are same, so return existing PG object/instance
return pg
# Current and desired config are different, so reconfigure the PG
new_spec = self.__compute_reconfigure_portgroup_spec(
pg_name, pg.config.configVersion, vlan_id, mac_learning_enabled, uplink_port_order)
self.__reconfigure_portgroup(pg, new_spec)
else:
# PG doesn't exist, so create it
pg_spec = self.create_port_group_spec(pg_name, vlan_id, mac_learning_enabled, uplink_port_order)
self.add_portgroup(pg_spec)
return self.get_portgroup(pg_name)
- Inside the if condition to check for the port group, add a return statement to return the port group immediately: return pg The code snippet should look like this after this modification:
def create_or_modify_portgroup(
self, pg_name: str, vlan_id: int, mac_learning_enabled: bool,
uplink_port_order: UplinkPortOrder) -> Optional[vim.dvs.DistributedVirtualPortgroup]:
"""
This method creates the specified port group if it is not present.
If it is already present, then it is modified according to the specified configuration.
@param pg_name: The name of the port group to be created/modified.
@param vlan_id: The value for VLAN. If 0 is specified, then it means no VLAN.
@param mac_learning_enabled: Whether to enabled mac learning or not.
@param uplink_port_order: The grouping and ordering of the uplinks.
@return The process port group
@rtype Optional[vim.dvs.DistributedVirtualPortgroup]
"""
pg = self.get_portgroup(pg_name)
if pg:
return pg
# PG already exists, so check if its current configuration is same as desired configuration
existing_pg_spec = self.__compute_comparison_spec_from_pg(pg)
desired_pg_spec = self.__compute_comparison_spec(vlan_id, mac_learning_enabled, uplink_port_order)
# Complex Named Tuples can still be directly compared for equality
if existing_pg_spec == desired_pg_spec:
# Current config and desired config are same, so return existing PG object/instance
return pg
# Current and desired config are different, so reconfigure the PG
new_spec = self.__compute_reconfigure_portgroup_spec(
pg_name, pg.config.configVersion, vlan_id, mac_learning_enabled, uplink_port_order)
self.__reconfigure_portgroup(pg, new_spec)
else:
# PG doesn't exist, so create it
pg_spec = self.create_port_group_spec(pg_name, vlan_id, mac_learning_enabled, uplink_port_order)
self.add_portgroup(pg_spec)
return self.get_portgroup(pg_name)
- After adding the return statement, the rest of the code inside the if statement block is unreachable, it can be left as-is or removed (optional).
- Save the file and exit VIM
- Exit the docker container with the ‘exit’ command.
- Restart the docker container with the following commands:
systemctl restart tcf-manager-deploy
systemctl start tcf-manager
- After the above restart command returns control, list the running docker containers with the following command and ensure that the container named “tcf-manager’ is up and running:
- Any hosts added from this point forward can be provisioned successfully without being affected by this issue. Any hosts that failed previously due to this issue will now be provisioned successfully after a resync.
Additional Information
Impact/Risks:
Impacts TCA 2.3.
Impacts TCA 2.3.
Feedback
Yes
No
Powered by
