DFW Firewall rule(s) are not "stitched" or "attached" to the flows in VCF Operations for Networks
search cancel

DFW Firewall rule(s) are not "stitched" or "attached" to the flows in VCF Operations for Networks

book

Article ID: 403645

calendar_today

Updated On:

Products

VCF Operations for Networks

Issue/Introduction

DFW Firewall rule(s) are not "stitched" or "attached" to the flows in VCF Operations for Networks.

In normal operation, when flow data is received by Collector node(s), it is distributed among different nodes using a "sharding" technique. 

After processing, when the data is consolidated into the VCF Operations for Networks database, the "shards" are then "stitched" back together from the Platform node(s) perspective for display in the GUI. 

In VCF Operations for Networks, DFW IPFix is enabled on the respective NSX manager datasource and flows are seen but DFW Firewall rule(s) are not observed in the displayed flow information.

  1. Below is an example flow screenshot from VCF Operations for Networks:





  2. Firewall rule inventory is visible on VCF Operations for Networks GUI when used query with ruleid as seen in below screenshot:




    On investigation, the raw flows can be seen in the VCF Operations for Networks database using command on the collector node:
    nfcapd -ip src_ip, dst_ip -l 5

    Above command should list 5 raw nfcapd files and raw data will be displayed where firewall rule id information can be seen.


  3. From collector support bundle when collector logs are reviewed at location /var/log/arkin/collector the information for the DFW IP Rules can be observed as well:

    grep -v "policymanager.utils.PolicyManagerUtils|dataprovider.utils.CollectorUtils" collector.STDOUT-2025-07-01-15.00.47.log.error | grep <DFW_RULE_NAME> | grep "Fetching Realized entities" | cut -d ' ' -f4,10 | sort | uniq -c | less

    4 NSXT_<FQDN_OF_NSXT_MANAGER>_Config_OpMgr_Policy-1 managerId=Policy.Onprem.<FQDN_OF_NSXT_MANAGER>,
    1 NSXT_<FQDN_OF_NSXT_MANAGER>_Config_OpMgr_Policy-1 managerId=Policy.Onprem.<FQDN_OF_NSXT_MANAGER>,
    6 NSXT_<FQDN_OF_NSXT_MANAGER>_Config_OpMgr_Policy-1 managerId=Policy.Onprem.<FQDN_OF_NSXT_MANAGER>,
    3 NSXT_<FQDN_OF_NSXT_MANAGER>_Config_OpMgr_Policy-1 managerId=Policy.Onprem.<FQDN_OF_NSXT_MANAGER>,
    9 NSXT_<FQDN_OF_NSXT_MANAGER>_Config_OpMgr_Policy-1 managerId=Policy.Onprem.<FQDN_OF_NSXT_MANAGER>,
    2 NSXT_<FQDN_OF_NSXT_MANAGER>_Config_OpMgr_Policy-1 managerId=Policy.Onprem.<FQDN_OF_NSXT_MANAGER>,
    19 NSXT_<FQDN_OF_NSXT_MANAGER>_Config_OpMgr_Policy-1 managerId=Policy.Onprem.<FQDN_OF_NSXT_MANAGER>,

    grep -v "policymanager.utils.PolicyManagerUtils|dataprovider.utils.CollectorUtils" collector.STDOUT-2025-07-01-16.00.54.log.error | grep <DFW_RULE_NAME> | grep "Fetching Realized entities" | cut -d ' ' -f4,10 | sort | uniq -c | less

    3 NSXT_<FQDN_OF_NSXT_MANAGER>_Config_OpMgr_Policy-1 managerId=Policy.Onprem.<FQDN_OF_NSXT_MANAGER>,
    1 NSXT_<FQDN_OF_NSXT_MANAGER>_Config_OpMgr_Policy-1 managerId=Policy.Onprem.<FQDN_OF_NSXT_MANAGER>,
    1 NSXT_<FQDN_OF_NSXT_MANAGER>_Config_OpMgr_Policy-1 managerId=Policy.Onprem.<FQDN_OF_NSXT_MANAGER>,
    3 NSXT_<FQDN_OF_NSXT_MANAGER>_Config_OpMgr_Policy-1 managerId=Policy.Onprem.<FQDN_OF_NSXT_MANAGER>,
    3 NSXT_<FQDN_OF_NSXT_MANAGER>_Config_OpMgr_Policy-1 managerId=Policy.Onprem.<FQDN_OF_NSXT_MANAGER>,
    10 NSXT_<FQDN_OF_NSXT_MANAGER>_Config_OpMgr_Policy-1 managerId=Policy.Onprem.<FQDN_OF_NSXT_MANAGER>,
    3 NSXT_<FQDN_OF_NSXT_MANAGER>_Config_OpMgr_Policy-1 managerId=Policy.Onprem.<FQDN_OF_NSXT_MANAGER>,
    22 NSXT_<FQDN_OF_NSXT_MANAGER>_Config_OpMgr_Policy-1 managerId=Policy.Onprem.<FQDN_OF_NSXT_MANAGER>,

    Rule not handled error is see below from collector logs as location

    2025-07-01T15:00:59.701Z ERROR tasks.datafetchers.PolicyManagerConfigTask NSXT_<FQDN_OF_NSXT_MANAGER>_Config_OpMgr_Policy-2 fetchEntities:363 Object type Rule not handled.
    2025-07-01T15:06:22.757Z ERROR tasks.datafetchers.PolicyManagerConfigTask NSXT_<FQDN_OF_NSXT_MANAGER>_Config_OpMgr_Policy-0 fetchEntities:363 Object type Rule not handled.
    2025-07-01T15:07:24.539Z ERROR tasks.datafetchers.PolicyManagerConfigTask NSXT_<FQDN_OF_NSXT_MANAGER>_Config_OpMgr_Policy-1 fetchEntities:363 Object type Rule not handled.
    2025-07-01T15:09:15.637Z ERROR tasks.datafetchers.PolicyManagerConfigTask NSXT_<FQDN_OF_NSXT_MANAGER>_Config_OpMgr_Policy-2 fetchEntities:363 Object type Rule not handled.
    2025-07-01T15:09:26.112Z ERROR tasks.datafetchers.PolicyManagerConfigTask NSXT_<FQDN_OF_NSXT_MANAGER>_Config_OpMgr_Policy-0 fetchEntities:363 Object type Rule not handled.
    2025-07-01T15:09:30.435Z ERROR tasks.datafetchers.PolicyManagerConfigTask NSXT_<FQDN_OF_NSXT_MANAGER>_Config_OpMgr_Policy-0 fetchEntities:363 Object type Rule not handled.
    2025-07-01T15:11:03.418Z ERROR tasks.datafetchers.PolicyManagerConfigTask NSXT_<FQDN_OF_NSXT_MANAGER>_Config_OpMgr_Policy-1 fetchEntities:363 Object type Rule not



  4. Validate on which vCenter(s) are associated with a given NSX-T Manager and then go to database on the platform node to validate NSX Manager and VC Manager relation in vRNI DB, you will see it is incorrect when checked via rdb command such as below:

    Run following rdb commands in any platform node this will show you the mk_ids assigned by VCF Operations for networks to the respective vCenter and NSX-T Manager.  Please note this can only be ran LIVE on the node.

    search -q <NSX-T manager FQDN/IP> | traversal -pt associatedManagers

search -q <NSX-T policy manager FQDN/IP> | traversal -pt associatedManagers

search -q <NSX-T policy manager FQDN/IP> | traversal -pt vCenters

search -q <VC manager FQDN/IP> | traversal -pt associatedManager

    From above command ensure you swap the respective FQDN/IP for the vCenter and NSX-T Manager 

NOTE:  VCF Operations for Networks was formerly named Aria Operations for Networks (AON), and prior to that was named vRealize Network Insight (vRNI).

Environment

VCF Operations for Networks 6.14.0

Cause

Stale entries of mapping relation between vCenter Manager and NSX Manager within Aria Operations for Networks database is seen, although the raw data see the rule ID was coming in.

The Stale entries for vCenter and NSXT Mapping relation caused the issue where DFW Rules are not stitched or attached  to the flows in VCF Operations for Networks GUI.

Resolution

VCF Operations for Networks team is working on understanding the root cause for this.

This is a condition that may occur in VCF Operations for Networks 6.12, 6.13, 6.14 and VCF Operations for Networks 9.0 

For now an interim Workaround is available only for VCF Operations for Networks version 6.14 version where GSS would need to perform a cleanup the mappings via custom script on the database of VCF Operations for Networks.


Platform Node(s) service restart is required.  

For Workaround with version 6.14.0, Open a support case with Broadcom Support and refer to this Knowledge base Article, so that support team can review your VCF Operations for Networks deployment and make changes as needed. For more information, see Creating and managing Broadcom support cases.

Additional Information

If you are contacting Broadcom support about this issue, please capture below when Opening the Case:

  1. The details( Screenshots and outputs)  of all the 4 symptoms as mentioned in Issue/Introduction.

  2. Support bundle from all the VCF Operations for Networks Platform appliances and only the Collector appliance where the vCenter and NSXT Manager are added as datasource.
    Steps to generate support bundle for VCF Operations for Networks Appliances are mentioned in Broadcom Knowledge Base Article: https://knowledge.broadcom.com/external/article/343485

Attachments

cleanup.tar.gz get_app
Powered by Wolken Software