Troubleshooting duplicate assets

Products

• IT Management Suite
• Asset Management Solution

Issue/Introduction

How to better understand and troubleshoot duplicate assets?

Environment

ITMS 8.x

Resolution

This white paper article provides general information and best practices on how to understand, prevent and troubleshoot duplicate assets. Note: This article is primarily intended for CMDB Solution's duplicate assets and merges. Most references are therefore regarding these asset types, not other non-CMDB duplicate assets and merges, which are only briefly discussed.

Other non-CMDB duplicate assets and merge types therefore are only briefly covered.

What Are Duplicate Assets and How Are They Caused?

The phrase "duplicate assets" refers to the situation when two or more asset records share at least one data class value between them, such as the resource name or serial number. These asset records may or may not refer to the same physical asset. In general, this refers to one of the following:

• A CMDB Solution record has a duplicate value to another CMDB Solution record, such as a duplicate serial number.
• A CMDB Solution record has a duplicate value to an Inventory Solution record, such as a duplicate system number.
• An Inventory Solution record has a duplicate value to another Inventory Solution record. Note: This sort of duplicate is rare and is not resolved by performing merges. For example, if there are duplicate records for the same GUID in the Inv_HW_Computer_System table that is caused by issues from Scenarios 4 or 5 below, or other causes. These must be cleaned up in a special way, such as by running the "Remove duplicate Inventory records.txt" SQL script.

Duplicate assets can be any asset type but are most often associated with duplicate computer records. Depending on how the computers and their data classes were brought into the Symantec Management Platform often determines how the duplicates were caused. For example, the following are three scenarios where duplicate computers can be caused:

Scenario 1: Duplicate computer names (normal)

Computers that have duplicate resource names are not considered to be an issue because the resource name data class by itself is not required to be unique. These can be created in a variety of ways, such as by the following:

• Computers were received by an Asset Management Solution purchase order, twenty may all be named "Dell Inspiron 15".
• Computers were scanned and uploaded by Barcode Solution, fifteen may all be named "New Computer". 

Scenarios 2 and 3: Duplicate unmanaged and managed computer records (normal) and running the Inventory to Asset Synchronization task (normal)

A common business practice is to create unmanaged "staging" computer records before their physical computers are brought into the Symantec Management Platform. Later, their physical computers are integrated with a Symantec Management Agent and this creates the managed computer records. These processes result in one unmanaged computer record and one managed computer record both referring to the same physical computer. The following example demonstrates how duplicate assets can be created by unmanaged and managed computer records. Note: Computer records created by Asset Management Solution, CMDB Solution, Barcode Solution or Data Connector are all considered to be unmanaged records with only CMDB Solution/Asset Management data classes initially. Computer records created by a Symantec Management Agent checking in for the first time are considered to be managed records with only Core data classes initially.

• A Data Connector import rule is ran to import in ten HP Pavilion 500 computers from an Excel file. The import includes, for example, the computer's serial numbers. These records become unmanaged computer records and do not yet refer to any physical computers. Note: It is recommended to create staging records with either the serial number and/or the system number. Once the managed computer record checks in with this information, a merge task can easily then combine the records together. But without these merge keys, it can be difficult if not impossible to logically determine which staging records should match up and merge to a managed record, resulting in the user having to do so manually.
• Later, the ten HP Pavilion 500s are purchased, imaged, plugged into the network and integrated with a Symantec Management Agent. These then create managed computer records.
• Lastly, the Inventory to Asset Synchronization task is ran. (This results in the 3rd scenario.) As the unmanaged computer records already have the serial numbers, when the task runs, the managed computer records now also contain the same serial numbers. Note: Generally, most Asset product functions are incapable of creating duplicates that are unique data classes. For example, a user cannot manually edit two computers and give them the same Serial Number or the same System Number value. However, the Inventory to Asset Synchronization task can cause this, if Inventory Solution has duplicate values (refer to scenario 5, below). While duplicate assets are often thought of as caused by Asset products, this is therefore incorrect, but the end result and symptoms are usually first detected when using Asset products. The root cause is nearly always caused by a different product, however.
• The total end result of scenarios 2 and 3 are duplicate asset records both referring to the same physical computers and duplicate data class values for each one's serial number.

Scenarios 4 and 5: Reusing or renaming computer names to use old computer names (abnormal)

A computer has a name that is desired to be reused. This may occur after the old computer is retired and a new one then reuses the old name. This can result in duplicate IP addresses or MAC addresses, among other duplicate issues. The following examples demonstrate how these can occur. WARNING: It is not recommended that users reuse or rename computers to use an old computer name that already exists in the Symantec Management Platform database as this will result in serious issues throughout the Symantec Management Platform. Duplicate computer records will occur as well. For more information on troubleshooting this issue, refer to the following article:

Computer's Manufacturer, Model, Serial Number or System Number are incorrect or missing
158254

• A computer is integrated with a Symantec Management Agent and it checks in. This creates an asset record with the computer's name. This computer is later retired by a user. Later still, a different computer is integrated with a Symantec Management Agent and checks in. However, this computer reuses the old computer's name. This results in two different computer records using the same computer name referring to two different physical computers.
• A computer's name is renamed to be the same name as a computer that already exists in the Symantec Management Platform database.
• A deployed image includes the same name as a computer that already exists in the Symantec Management Platform database.

If the Inventory to Asset Synchronization task is running when there is duplicate Inventory records, this then causes CMDB Solution duplicates for its Manufacturer, Model, Serial Number and/or System Number values, depending on what is duplicated in the Inventory Solution records. (This is the 5th scenario.) When this occurs, this causes a domino effect of duplicates which starts with the process, continues to Inventory Solution data classes and finally ends up also in CMDB Solution data classes. Symptoms include being unable to save changes to affected records (for the Serial Number and System Number duplicates only), as duplicate unique values cannot be resaved. The preferred way to resolve this is to correct the practice that is causing the duplicate Inventory Solution data classes, clear out the incorrect and duplicate CMDB Solution data classes (such as with a CMDB rule), and then finally, rerun the Inventory to Asset Synchronization task to update CMDB Solution's data classes with the correct values from the Inventory Solution data classes. WARNING: It is not recommended to run a merge task to resolve these duplicates, as the most recent record will be the one that is kept, which in this case, is usually the duplicate.

Scenario 6: Reusing IP addresses or MAC addresses

As discussed in scenario 4, reusing computer names can also result in duplicate IP addresses or MAC addresses. This can also be caused by:

• Reusing a virtual machine that has the same IP address or MAC address.
• Deploying for use a virtual machine that has the same IP address or MAC address.
• Reimaging computers such as by using Deployment Solution.

For VMs, if care is not taken to ensure that each one has its own unique IP address and especially MAC address, this can result in duplicates or in automatic merges between two different computers. In some situations, this may be normal. For example, if an old VM has a specific MAC address and this is reused to be a new VM but retains its original MAC address, this can result in to computer records in Altiris but with the same MAC address.

Normally, computers with duplicate MAC addresses are found and instantly merged when the new one checks in, but in some cases they aren't. In some cases these are not able to be merged and remain unresolved. Unfortunately there is no merge rule other that CMDB Solution offers to be able to further attempt to merge records like this, at least not using the MAC address as the merge key. If there is another key field that can be used, then a CMDB merge could then merge these records. If, however, the user doesn't want them to be merged but these are two different computers, the only solution is to fix the root cause: correct the MAC address on one of the computers so that it unique. If the future, do not force the virtualization software to use a static MAC address and do not reuse VMs that have the same MAC address (if they are functionally different computers then) as either will result in this issue reoccurring.

Once the VM has unique MAC address, when its Symantec Management Agent checks back in, this will then be updated in the Altiris database. If the VM does not have a Symantec Management Agent, then this would need to be manually changed in SQL instead, specifically, in the Inv_AeX_AC_TCPIP table.

Scenario 7: Network Discovery

Using Network Discovery, in some configurations, can result in duplicate computer records. Often it will catch this and automatically merge the two records together. When this fails to do so, generally the following are what causes the two records to be not merged:
 

• The FQDN is slightly different for each computer. For example, one may be my.domain.com, another may be my.domain. This will fail to see this as the same FQDN and therefore not merge. This can be due old DNS information.
• Network Discovery is used to discover new computers. When a computer's Symantec Management Agent checks in later, this can result in a duplicate. Note: The recommended way to discover new computers is to use the Microsoft Active Directory Import, not Network Discovery.

Shared and Duplicate GUIDS

If asset records share the same GUID, or, a GUID is somehow duplicated many times in a table, these become very serious issues. While these could be thought of as duplicate assets, and in fact the Resource Merge Rule can merge duplicate GUIDs, they are seen as a different type of issue and are not otherwise covered in this article. For information on how to troubleshoot these, refer to the following article:

Detecting computers with Shared GUIDs
180432

How to Find Duplicate Assets

Duplicate assets can be found by a variety of ways, such as when using reports, editing assets, SQL scripts or by searching for data such as for a computer's serial number. Three out of box reports also help find duplicate assets, which can be found under Reports > Service and Asset Management > Merge. These three reports list assets that have duplicate CMDB Solution barcode, serial number and system numbers, respectively.

Another way is attempting to edit an asset that has a unique data class that has an existing duplicate, such as the Serial Number or System Number values. For example:

Example 1: Two (or more) computers have duplicate unique data class values already, which were caused by unknown processes.

• Two computer records include the same System Number values.
• The user edits one of these records and makes changes to a different field, such as the Comment.
• The user attempts to save the record; an error occurs stating that it cannot be saved because the System Number is a duplicate even though the user, themselves, didn't add this into the record, it still exists when opened, and cannot be re-saved because of this, even with changes to a different field.

Example 2: The user tries to give another computer a duplicate unique data class value that another computer already has. This can be unknowingly or by editing a computer that already has this issue.

• A computer record has a specific System Number value.
• The user edits a different computer record and tries to add the same System Number value to it.
• The user attempts to save the record; an error occurs. (This is the exact same error from Example 1.)

As the error states, there is another record that already has the same System Number data class value, and therefore, the record being edited, cannot be saved (regardless of the changes made, unless that was to remove/change the System Number to be unique).

Refer to the following article that further discusses this error for additional information about it:

Error "An error occurred saving changes. A resource with a duplicate field already exists in the system." occurs when trying to save a computer
161062

Included as attachments in this article are several SQL scripts that can find duplicate assets. These are:

• Find duplicate CMDB serial numbers.txt - This SQL script finds duplicate CMDB Solution serial numbers by reviewing records from CMDB Solution's Inv_Serial_Number.[Serial Number] table and data class.
• Find duplicate CMDB system numbers.txt - This SQL script finds duplicate CMDB Solution  system numbers by reviewing records from CMDB Solution's Inv_Identity.[System Number] table and data class.
• Find duplicate CMDB barcode numbers.txt - This SQL script finds duplicate CMDB Solution barcode numbers by reviewing records from CMDB Solution's Inv_Barcode.[Barcode] table and data class.
• Find duplicate computer names.txt - This SQL script finds duplicate computer names by reviewing records from RM_ResourceComputer (or RM_ResourceVirtual_Machine) and vItem.
• Find matching LIKE computer names.txt - This SQL script finds duplicate computer names that are similar to other computer names. While this is technically therefore not actually showing duplicate computer names, if similar names are used and are seen as duplicates, such as similar naming conventions used for staging and managed computers, this can help identify these that should be merged. However, merge tasks cannot merge these as there are no actual duplicate names found from this process.
• Find duplicate Inventory serial numbers.txt - This SQL script finds duplicate Inventory Solution serial numbers by reviewing records from Inventory Solution's Inv_HW_Computer_System.[Identifying Number] table and data class. Note: There is no merge process that can merge duplicate assets that are only found in Inventory Solution managed records. Merge tasks are intended to only merge two or more unmanaged records or one unmanaged record with a managed record. The preferred way to resolve this is to ensure that the Inventory Solution plug-in is installed on the affected computers and a hardware inventory policy has ran to update their incorrect inventory data.
• Find duplicate Inventory system numbers.txt - This SQL script finds duplicate Inventory Solution system numbers by reviewing records from Inventory Solution's Inv_SW_BIOS_Element.[Identification Code] table and data class. Note: There is no merge process that can merge duplicate assets that are only found in Inventory Solution managed records. Merge tasks are intended to only merge two or more unmanaged records or one unmanaged record with a managed record. The preferred way to resolve this is to ensure that the Inventory Solution plug-in is installed on the affected computers and a hardware inventory policy has ran to update their incorrect inventory data. Alternatively, the next SQL script can be ran to help clean these up as well.
• Find duplicate IP addresses.txt - This SQL script finds duplicate IP addresses, which are usually caused by reusing computers or computer names.
• Find duplicate MAC addresses.txt - This SQL script finds duplicate MAC addresses, which are usually caused by reusing computers or computer names.
• Remove duplicate Inventory records.txt - This SQL script helps clean up duplicate Inventory Solution duplicate records, such as from the Inv_HW_Computer_System table.

How to Find Merged Assets

While it is not technically the scope fo this article to answer, sometimes it's useful to see what assets have already been merged. Included as an attachment in this article is a SQL script that can do this, which is "Find CMDB merged assets.txt".

How to Merge Duplicate Assets

Duplicate assets can be resolved in one of three ways: by resolving the conflict by editing or otherwise changing the duplicated shared data class values, by deleting the unwanted duplicate or by merging the duplicates together. It's usually easiest to perform an automatic merge of the duplicates to resolve the issue, of which CMDB Solution provides three tasks (described below) that can perform this.

Merge types under control by the user include a manual merge and several automatic merge tasks, all of which are controlled by CMDB Solution. For automatic merge tasks, however, the user should be very cautious about running these as they can delete records that were not intended to be deleted (from the merge process). For all CMDB merges, the latest record is kept, along with the latest updated data classes from each of the records. Certain resource merge keys can cause the wrong record then to be kept, such as if Resource Name were used. If the oldest is the record to keep, and duplicate names were added later (which can be a normal process in the work environment), the correct old record is deleted and merged in favor of the newest, which may not be what the customer expects. For example both Barcode Solution and Asset Management Solution can receive new computers as a single name, such as "New Computer". If an automatic merge task were ran against Resource Name, all "New Computer" records except the latest, last one, are deleted and merged with it; not at all what should be done as the actual valid assets are now permanently deleted. If this were then performed without understanding this, the user would then find that they have "accidentally" deleted a lot of valid assets, with no way to restore these other than re-receiving them or restoring from a database backup.
 
CMDB Solution Merges
 
CMDB Solution merge tasks are found under Manage > Jobs and Tasks > Service and Asset Management > CMDB. Note: For a merge task to work successfully, the target records must contain matching duplicate data class values. For example, if Resource Name is used as the merge key, but no duplicate resource names are found but similarly named ones are (for example, TestPC1, Test PC2), these cannot be merged with a merge rule as there are no actual duplicate data class values.

All CMDB merge types, manual or merge task, always keep the latest record and all of the latest data class values of the records to merge. (With the exception of the manual merge, which the user can control which record and data classes are kept.) For example: New Computer (1) was created on 5/1/2014, and New Computer (2) was created on 7/1/2014. New Computer (2) would be kept, and (1) would be deleted. If (1) had data classes that (2) didn't have, they are copied over to (2). Or likewise, if (1) had data classes that were filled out that (2) had, but were more recent, these will overwrite (2)'s as they are newer.

The following are merges that are controlled by CMDB Solution:

• Merge Duplicate Computers. This task merges duplicate unmanaged computer records, or, duplicate unmanaged computer records to duplicate managed computer records by the merge keys of Barcode, Serial Number or System Number. For example, two CMDB Solution records may have the same serial number as found in the Inv_Serial_Number.[Serial Number] table and data class and are therefore eligible to be merged by this task. Or, one CMDB Solution record has the same serial number in the Inv_Serial_Number.[Serial Number] table and data class as an Inventory Solution record has in its Inv_HW_Computer_System.[Identifying Number] data class and are therefore eligible to be merged by this task. Note: If Barcode is the merge key, this then only applies to two unmanaged computer records, as Inventory Solution does not have a Barcode data class. 
• Merge Duplicate Users. This task merges duplicate user records by the merge keys of UserID and Domain or by Given Name and Surname.
• Resource Merge Rule. This task can merge virtually any asset type with the same type and works in the same way for computers as the Merge Duplicate Computers task does, but with the added ability of providing the additional merge keys of Resource Name, Resource Guid and Computer Name/Domain.
• Manual merge. A manual merge can be performed on two records at a time. To do this, go to any asset report where the duplicates appear a, hold down the Ctrl key and click on the two records. Then, right click on the selected records and choose Merge Resources. By default, the newest record and newest data classes between the two records are kept. The user can choose to override these, however.

Barcode Solution Merge

Barcode Solution can also be configured to auto-merge assets when they are synched from a handheld scanner into the Altiris database. This is not commonly used, but could be enabled and causing "unexpected merges" to occur, therefore. This is located under Settings > All Settings > Service and Asset Management > Barcode Solution Settings > Merge Settings. By default, only Barcode is selected to be a merge key.

Other Merge Types

There are two other merge types, an automatic Core forced merge (controlled by the Symantec Management Platform Core system) and an automatic Network Discovery forced merge:

• Merge Duplicate Resources automatic internal tasks from Core. This is automatically scheduled to run every day at 6:15 AM by the Symantec Management Platform via Windows Scheduled Tasks. (This task is not controlled by CMDB Solution.) This only merges duplicate managed computer records if they have a duplicate FQDN, Distinguished Name or MAC Address. For example, a managed computer record has the resource name of "Dell Server". A user then changes the computer's name on the server itself. When the computer's Symantec Management Agent later checks in, a second managed computer record is created with the new computer name. At 6:15 AM the next day, the Merge Duplicate Resources task is ran. The task finds that the MAC address is the same for both records and merges them. This task can also fire immediately when the new changed managed asset record checks in, forcing an automatic merge.
• Network Discovery internal automatic merge. If assets are being brought in by Network Discovery, this can automatically instantly merge them with what it detects are duplicate assets, similar to how Core may do so. To troubleshoot this, if this is not desired, ensure that the assets do not have the same name or IP address otherwise a forced automatic merge may occur.

General Merge Notes

• Merging duplicate assets does not resolve the reasons why they were caused. The user must determine what the causes were still and resolve these otherwise new duplicate assets will likely occur in the future. While CMDB Solution provides the means (the merge tasks) to resolve duplicate assets, generally speaking, Asset Management Solution, CMDB Solution, Barcode Solution and Data Connector are incapable of creating duplicate assets and are not normally the actual cause of the issue.
• Merge tasks can merge any number of duplicate records found into a single, unified record. Manual merges can only merge two records at a time, however.
• Merge tasks, except the Merge Duplicate Resources automatic internal task, are not automatically scheduled to run; the user must manually run these or schedule them to run.
• CMDB merges do not take effect (fire) immediately. If assets are unexpectedly being merged as soon as they appear, this is not caused by CMDB merges, but is likely due to the above mentioned Core or Network Discovery automatic merge. CMDB merges are manual or run on a schedule or by demand of a user, and therefore cannot work "immediately" upon the Notification Server detecting that duplicates are present.
• Only manual merges enable the user to override which record is kept and what data classes are kept. Automatic merges always keep the most recent record and most recent data classes (if they contain data).
• No merge type can merge dissimilar asset types together. For example, monitors cannot be merged with cell phones, computers cannot be merged with virtual machines. (While virtual machines are a subset of the computer asset type, they are different types and therefore cannot be merged together.)
• Merging duplicate assets based on the merge key of the Serial Number or System Number only evaluate CMDB Solution's Serial Number or System Number and Inventory Solution's Serial Number or System Number. While other "serial number" and "system number" data classes are present from other products, these are not evaluated and if these are duplicated, merges do not take these into account.

The CMDB Task Inventory to Asset Synchronization

The Inventory to Asset Synchronization task is often mistaken for a merge, which it is not. The purpose of this task is to copy Inventory Solution's Manufacturer, Model, Serial Number and System Number to the same record's CMDB Solution's matching data classes. This task can be used after a merge that has combined an unmanaged and managed record together or it can be used to enable a merge task to then work based on the merge key or Serial Number or System Number. For more information about this task, refer to the following article:

How does the CMDB task "Inventory to Asset Synchronization" work?
181655

Troubleshooting Task Issues

Occasionally, there may be an issue with the merge task. If the Task Server is functioning fine and NSEs are being processed, however, this should not normally be an area of concern. If it is suspected the task is the issue and is not working, review the Altiris server logs to see if any errors are occurring and if the task is firing successfully. If everything appears fine, then most likely if the expected duplicate assets were not merged was because they did not meet the above criteria on how the merge tasks work.

Best Practice When Using Unmanaged and Managed Computer Records

When unmanaged computer records are to be used, is it recommended that these include a key value, such as a serial number or system number. This easily enables them to be later merged using an automatic merge task once Inventory Solution has ran its hardware inventory policy on the managed computer records. If this is not done, however, there are no matching values to find between the two records. It can be difficult then to spot these in the Console as there is no common shared data class value to match them up with. A manual merge would then be needed to merge the two records together.

Finding the Root Cause of Duplicate Assets

By evaluating this article's information, often the root cause of duplicate assets can be understood, which can often by the processes that the user has set up to bring assets into the database. It's it not clear how an asset was created, review its Resource Change History. this can be found by right clicking on an asset in a report and then selecting CMDB Functions > Resource Change History. This lists most changes to an asset including its creation and changes to its data classes.

Related Article

Unable to find computer by searching for its Serial Number
161236