Why do I see so many rows returned in a lookback API query?

Products

• Rally SaaS

Issue/Introduction

When performing a query in lookback API (LBAPI) a bunch of extra rows are returned that appear to be duplicates of each other.

Environment

Release : SAAS

Cause

When a work item is updated, a new snapshot is taken with the current state of all fields, regardless of what was updated.  

So if you have a query that is looking for items in a particular State, for example, that particular State value will be present regardless of any of the other fields that were updated to cause that snapshot to be taken.  So as long as that State is still valid for all of those snapshots, you'll see representations of those records even though you're only interested in State changes.

Resolution

A way to mitigate this is to also look in the _PreviousValues object.  The _PreviousValues object contains only things that changed for that particular snapshot.  So using the State example, you can find "_PreviousValues.State": { $exists: true} which indicates that State changed.  This would cut down on the seemingly duplicate records.