Smarts NCM: Reinstalling a Device Server (DS) fails, but the Smarts NCM console interface and cflist command output show duplicate Device Servers; How to remove Duplicate Device Server from cflist

Products

• VMware Smart Assurance

Issue/Introduction

Symptoms:


Smarts NCM Device Server (DS) crashes

The following conditions, events or system behaviors may be observed with this issue:

• Reinstalling and re-registering a Smarts NCM Device Server fails 
• Cannot run a job in Smarts NCM because there are duplicate Device Servers
• Smarts NCM console and cflist command output show duplicate Device Server entries

Environment

VMware Smart Assurance - NCM

Cause

This issue occurs when the Device Server was not removed from the environment correctly. Because of this, when trying to reinstall/re-register the Device Server, a duplicate entry for the DS gets created in the InfraDB.

Resolution

If you encounter this issue in your environment, do the following:

1. Run the following SQL query on the Smarts NCM DB:

select * from cm_device_server;

2. Review the output of the above command for duplicate Device Server (DS) entries in the Postgres database. If there a duplicate entry is found, see EMC KB article " target="_blank"> Link Error: for instructions on how to remove the duplicates.
3. Once all duplicates are confirmed as removed from the Postgres DB, use the following command sequence on the infraDB on the Application Server (AS) to write out the cflist command output to a file (cflist.out):

service vcmaster stop

source /etc/voyence.conf

$VOYENCE_HOME/cgi-bin/cflist.cgi > cflist.out

4. Open the cflist.out file and analyze it within the shell using the vi editor, or copy the file to client machine and open it with a text editor such as Notepad.
5. Search the file for the hostname of the DS, and make note of the POP ID as in the following example for POP 3513:

POP 3513 "voyence_ds2" NetList= RsrcList=
(POP <ID> "<hostname>" NetList= RsrcList=)

6. Locate the additional POP entry for the given hostname, which should be the last POP entry. The last/second POP entry for the hostname is the duplicate hostname entry.
7. Ensure that you are at the second POP entry for the hostname (POP <ID> "<hostname>" NetList= RsrcList=).
8. Delete the last POP entry starting at POP <ID> and ending just before the following line:

NET 999 "Default Network" RsrcList= PopList= :

IMPORTANT! Do NOT delete the  NET 999  line.

9. Use the following command sequence to write back in the cflist into the infraDB on the AS:

$VOYENCE_HOME/cgi-bin/cwrite.cgi < cflist.out

service vcmaster start

10. Log into the Smarts NCM console and verify that the duplicate DS has been removed.
11. Proceed to install/register the Device Server.
12. Run a test job on a device that is managed with newly installed/registered DS.