DX NetOps Flow: Flows not displaying troubleshooting tips

Products

• Network Observability

Issue/Introduction

If you are having issues with Flows from DX NetOps Flow not appearing in the NetOps Portal Flow pages, please see this document to help troubleshoot your issues.

Environment

DX NetOps Flow

Resolution

1) First you want to confirm if the cluster receiving flows in 30995? 

=> tcpdump -i any port 30995

 

2) Is flow-proxy sending flows upstream to flow-collector-netflow-758c6d5fc5-8r9dg? (Check each of your nodes)

=> kubectl exec -it  flow-proxy-69v87 -- curl -s http://localhost:8080/actuator/prometheus | grep -e "^netflow" | sort

=> kubectl exec -it  flow-proxy-8jhp9 -- curl -s http://localhost:8080/actuator/prometheus | grep -e "^netflow" | sort

=> kubectl exec -it  flow-proxy-npfcd -- curl -s http://localhost:8080/actuator/prometheus | grep -e "^netflow" | sort

 

3) Is flow-collector-netflow-758c6d5fc5-8r9dg discarding flows because it cannot map the exporter IP address and interfaces ifIndex to an Item ID?

=> kubectl logs flow-collector-netflow-758c6d5fc5-8r9dg

log: portCache lookup failed. port key: {}

 

4) Is flow-inventory-lookup-58c86b9676-5njx8 retrieving inventory from the Data Aggregator and storing in the hazelcast cache?

=> kubectl logs flow-inventory-lookup-58c86b9676-5njx8

=> kubectl exec -it flow-inventory-lookup-58c86b9676-5njx8 -- curl http://localhost:8080/admin/cache/size?type=device

=> kubectl exec -it flow-inventory-lookup-58c86b9676-5njx8 -- curl http://localhost:8080/admin/cache/size?type=port

 

5) Is flow-collector-netflow-758c6d5fc5-8r9dg informing the Data Aggregator about the interfaces to populate the Flow administration page?

=> kubectl logs flow-collector-netflow-758c6d5fc5-8r9dg

log: Sending {} attribute update records to kafka topic

 

6) Is there any error in flow-collector-netflow-758c6d5fc5-8r9dg connecting to Kafka?

=> kubectl logs flow-collector-netflow-758c6d5fc5-8r9dg

 

7) Is the Data Aggregator properly connected to Kafka to consume the updates in step #5?

=> Check DA karaf.log and KafkaLog.log

=> Check Kafka Configuration in DA => curl http://da:8581/rest/kafkaconfig

 

8) Are the interfaces enabled in the Flow Administration page?

=> Check Portal's Flow Administration Page

 

9) Is there any error in flow-aggregator-56d8759957-gslsq connecting to Kafka?

=> kubectl logs flow-aggregator-56d8759957-gslsq

 

10) Are messages received in flow-aggregator-56d8759957-gslsq?

=> Use kafka-consumer utility connecting to 2-flow-data-record topic

 

11) Are messages received in flow-processor-55547c9cb8-vjm9j?

=> Use kafka-consumer utility connecting to flow-aggregated-record topic

 

12) Is there any error in flow-processor-55547c9cb8-vjm9j connecting to Vertica or Kafka?

=> kubectl logs flow-processor-55547c9cb8-vjm9j

=> kubectl exec -it flow-processor-55547c9cb8-vjm9j -- http://localhost:8080/actuator/prometheus

 

Please share your results with Broadcom Support if you are unable to resolve your issue on your own.