Smarts ESM: Disconnected LUN notifications still active after removing datastore and rediscovery of vCenter; Topology/Related objects Not Cleaned Up After ESX Removal; ESM not removing LUNs in OperationalState off

Products

• VMware Smart Assurance

Issue/Introduction

Symptoms:


Disconnected LUN notifications still active in SAM after removing datastore and rediscovery of vCenter:



There is a re-occurring issue when an ESX server is decommissioned and no longer reported by the vCenter. ESM correctly removes the ESX but leaves orphan objects behind such as Datastore and HostBusAdapter. This results in incorrectly alarm states.

The only solution is to manually delete the objects, either through the Console or ASL - re-discovery has no effect.
 

Notifications still displayed in Smarts SAM for LUNs, even after datastore is removed from vCenter and vCenter rediscovered

Environment

VMware Smart Assurance - SMARTS

Cause

This is a known bug exists in Smarts ESM

Resolution

This issue is resolved in the following releases:

Smarts 9.4 Patch 1 (9.4.0.1).
Smarts 9.3 Patch 10 (9.3.0.10).

Please upgrade to one of these versions to attain the fix.

if you cannot upgrade immediately, please find workaround below:

Workaround:
The below is the procedure to do a complete removal of the vCenter server in Smarts ESM v9.3 and its related objects from the topology.

Ideally, user needs to do the following if the vCenter host has been decommissioned:

a. Remove the vCenter credentials from the Smarts ESM.
b. Remove the host from the Smarts IP domain.
c. Remove the vCenter host from the Smarts ESM domain.
d. Discover All in the the Smarts ESM domain. This step does the clean up the remaining objects.

NOTE: The only thing that would still remain is the ScsiLuns even if its not having any ConnectDataPath. 
To remove the SCSILuns:

User needs to just sm_edit the esm-post.asl and uncomment lines 1972 - 1983 as below:

// 4) ScsiLun
count = 0;
lunList = getInstances("ScsiLun");
foreach lun (lunList) {

lunObj = object(lun);

if (!lunObj->isNull() && sizeof(lunObj->ConnectDataPath) == 0) {  

lunObj->delete();
count = count + 1;
 }

}
if (count > 0) {

print(time().ASLNAME." VMWARE_BOOKKEEPING: ".count." instance(s) 0f ScsiLun deleted for not connecting to any ScsiPath");
}
 

f. After the above step, the orphaned ScsiLuns in the Smarts ESM topology also would be cleared following a DiscoverAll in the the Smarts ESM domain.

This is the right way to remove a vCenter from the Smarts ESM domain.

Additional Information

If user is on ESM 9.0/ 9.1/ 9.2.x, the above shared steps wouldn't help, stale entries would remain in the domain, as there is a bug in cleanup of vCenter removal in that versions, which is fixed in Smarts ESM 9.3.