Upgrade failure going from 3.x to 20.2.1+

Article Id: 197201
Status: Published
Updated On: 14-08-2020 05:20
Products:
CA Performance Management - Usage and Administration , CA Infrastructure Management , DX NetOps
Issue/Introduction:

In 20.2.1, Data Aggregator moved to using PollableAttrs to store the Vendor Certification to use and Index to poll.  This was part of new feature to allow Metric Family item consolidation.
As part of 20.2.x upgrade, DA will read the current device components attributes for VC and Index, and write the new attributes to PollableAttrs facet.

If this doesn't happen successfully, users may see:

  1. Many components are listed as Filtered, when they should be Active.
  2. Running update MF throws an exception, and Polled Metric Families stays in Discovering status.

Exception is something like:

2020-08-12 10:17:19,6152020-08-12 10:17:19,615 | ERROR | ExceptionLog | A NEW application exception occurred (Key=3099f42881c0ef0b
107d279418e4a917f68761bf) : java.lang.NullPointerException | (ExceptionLogger.java:99)
java.lang.NullPointerException
        at java.lang.reflect.Array.set(Native Method)[:1.8.0_222]
        at com.ca.im.item.impl.repository.AppendIfAbsentWriteRequest.getModifiedIncomingValue(AppendIfAbsentWriteRequest.java:59)[
204:com.ca.im.item-repository.impl:20.2.2.RELEASE-317]
        at com.ca.im.item.impl.repository.WriteRequest.process(WriteRequest.java:282)[204:com.ca.im.item-repository.impl:20.2.2.RE
LEASE-317]
        at com.ca.im.item.impl.repository.GroupWriteRequest.process(GroupWriteRequest.java:80)[204:com.ca.im.item-repository.impl:
20.2.2.RELEASE-317]
        at com.ca.im.item.impl.repository.WriteRequest.verifyAndProcess(WriteRequest.java:257)[204:com.ca.im.item-repository.impl:
20.2.2.RELEASE-317]
        at com.ca.im.item.impl.repository.AttributeOperations.processItemWriteRequest(AttributeOperations.java:1154)[204:com.ca.im
.item-repository.impl:20.2.2.RELEASE-317]
        at com.ca.im.item.impl.repository.AttributeOperations.writeAttributes(AttributeOperations.java:1010)[204:com.ca.im.item-re
pository.impl:20.2.2.RELEASE-317]
        at com.ca.im.item.impl.repository.ItemRepositoryImpl.writeAttributes(ItemRepositoryImpl.java:1504)[204:com.ca.im.item-repo
sitory.impl:20.2.2.RELEASE-317]
        at com.ca.im.item.impl.repository.TenantItemRepository.writeAttributes(TenantItemRepository.java:355)[204:com.ca.im.item-r
epository.impl:20.2.2.RELEASE-317]
        at com.ca.im.tm.reconciliation.impl.AbstractReconciler.detectChangedAttrs(AbstractReconciler.java:530)[233:com.ca.im.aggre
gator.discovery:20.2.2.RELEASE-317]
        at com.ca.im.tm.reconciliation.impl.AbstractReconciler.findChanges(AbstractReconciler.java:517)[233:com.ca.im.aggregator.d
iscovery:20.2.2.RELEASE-317]
        at com.ca.im.tm.reconciliation.impl.AbstractReconciler.findChangedItems(AbstractReconciler.java:475)[233:com.ca.im.aggrega
tor.discovery:20.2.2.RELEASE-317]
        at com.ca.im.tm.component.discovery.impl.ComponentCreatorOnReadListener.doLocalReconciliation(ComponentCreatorOnReadListen
er.java:496)[233:com.ca.im.aggregator.discovery:20.2.2.RELEASE-317]
        at com.ca.im.tm.component.discovery.impl.ComponentCreatorOnReadListener.reconcileItems(ComponentCreatorOnReadListener.java
:383)[233:com.ca.im.aggregator.discovery:20.2.2.RELEASE-317]
        at com.ca.im.tm.component.discovery.impl.ComponentCreatorOnReadListener.onReadComplete(ComponentCreatorOnReadListener.java
:154)[233:com.ca.im.aggregator.discovery:20.2.2.RELEASE-317]
        at com.ca.im.item.impl.repository.DelegatingAttributeReadListener.notifyAttributeReadListener(DelegatingAttributeReadListe
ner.java:282)[204:com.ca.im.item-repository.impl:20.2.2.RELEASE-317]
        at com.ca.im.item.impl.repository.DelegatingAttributeReadListener.processSourceResponse(DelegatingAttributeReadListener.ja
va:103)[204:com.ca.im.item-repository.impl:20.2.2.RELEASE-317]
        at com.ca.im.item.impl.repository.DelegatingAttributeReadListener.onReadComplete(DelegatingAttributeReadListener.java:58)[
204:com.ca.im.item-repository.impl:20.2.2.RELEASE-317]
        at com.ca.im.dm.normalizer.impl.RawAttributeReadListener.onReadComplete(RawAttributeReadListener.java:308)[233:com.ca.im.a
ggregator.discovery:20.2.2.RELEASE-317]

 

Also, if the DA karaf.log(s) in /opt/IMDataAggregator/apache-karaf-x.x.x/data/log/ haven't rolled too much, you can check for this message to see if there were any errors during migration.

Start the migration of <count> devices.

Then search for:

Error occurred while migrating PCC of device: <itemid>
<Exception printed here>

Environment:

Upgrade from 3.x to 20.2.1+

Resolution:

Workaround:

1. Do a GET of http://DAHOST:8581/rest/discoverydefaultconfig to find item id.

2. Do a PUT to <http://DAHOST:8581/rest/discoverydefaultconfig/<ID>> with body:

<DiscoveryDefaultConfig version="1.0.0">
<PollableAttrMigrated>false</PollableAttrMigrated>
</DiscoveryDefaultConfig>

3. Restart the DA to get it to re-migrate the devices.

4. Run update MF on affected devices.