The hit count in Aria Operations for Networks on certain DFW rules is zero even though the NSX UI and Log Insight both register the hits

Products

• VMware vDefend Firewall
• VMware Aria Operations (formerly vRealize Operations) 8.x
• VMware vRealize Network Insight 6.x
• VMware vRealize Log Insight 8.x
• VCF Operations for Networks

Issue/Introduction

VCF Operations for Networks makes API calls to NSX to gather hit count statistics for DFW rules.

Some DFW rules for which the API calls made by VCF Operations for Networks to the NSX Manager return inconsistent results. 

For example, a VCF Operations for Networks query such as "firewall rules where Hit Count = 0" shows a number of different rule numbers, but doing a query in VCF Operations for Logs (formerly vRealize Log Insight) for one or more of the rule numbers shows that the rules are indeed being hit.

 

Example of a rule where the API call from VCF Operations for Networks shows a populated hit count for a specific rule ID:
/api/v1/firewall/sections/########-####-####-####-############/rules/####/stats

{
 "rule_id": "####",
  "packet_count": 270410579,
  "byte_count": 267475632508,
  "session_count": 965223,
  "hit_count": 1313037, <---------------------------------------------
  "l7_accept_count": 0,
  "l7_reject_count": 0,
  "l7_reject_with_response_count": 0,
  "popularity_index": 210,
  "max_popularity_index": 449742667,
  "max_session_count": 2801796992856,
  "total_session_count": 3684363757306,
  "_schema": "L3_FR"
}

 

Where:

########-####-####-####-############ is the Rule Section ID within NSX that contains the rule number; and

#### is the rule number

 

Example of a rule where the API call from VCF Operations for Networks shows a hit count of zero:

api/v1/firewall/sections/########-####-####-####-############/rules/stats
{
 "section_id": "########-####-####-####-############",
  "results": [
    {
     "rule_id": "####",
      "packet_count": 0,
      "byte_count": 0,
      "session_count": 0,
      "hit_count": 0, <--------------------------------------------- THIS IS NOT EXPECTED, AND ILLUSTRATES THE SYMPTOMS OF THE PROBLEM ------
      "l7_accept_count": 0,
      "l7_reject_count": 0,
      "l7_reject_with_response_count": 0,
      "popularity_index": 0,
      "max_popularity_index": 449800620,
      "max_session_count": 2802157023820,
      "total_session_count": 3684766913198,
      "_schema": "L3_FR"
    }
  ],
  "result_count": 1
}

 

NOTE:  VCF Operations for Networks was formerly named Aria Operations for Networks (AON), and prior to that was named vRealize Network Insight (vRNI).

Environment

All VMware NSX versions

Aria Operations for Networks 6.12
Aria Operations for Networks 6.12.1
Aria Operations for Networks 6.13
Aria Operations for Networks 6.14
VCF Operations for Networks 9.0

Cause

This is an issue affecting all NSX versions.

There is an issue with the query logic within NSX for the API:

api/v1/firewall/sections/<Section ID>/rules/stats

This causes VCF Operations for Networks to receive the erroneous data for individual rules.

Resolution

To confirm if you are encountering this issue:

1) Do a GET API to the NSX Manager of this form:

/api/v1/firewall/sections/<Section ID>/rules/<Rule ID>/stats

Where:

<Section ID> is the Section ID within NSX that contains the rule number; and

<Rule ID> is the rule number

NOTE:  To get the five tuple Section ID:

a) Do a GET API to the  NSX Manager of this form:

/api/v1/firewall/sections

b) In the output, search for the Section Name within NSX Manager for the rule number

 

 

2) If the reported Hit Count is non-zero, do the following GET API:

api/v1/firewall/sections/<Section ID>/rules/stats

 

3) If the data reported in this result for the rule ID "####" is zero, then this confirms the symptoms. 

 

This has been confirmed as a defect in NSX.  This issue is scheduled to be fixed in a future version of NSX.

 

There is no workaround.