{"id":21511502,"url":"https://github.com/netobserv/pano-research","last_synced_at":"2026-03-19T20:40:47.892Z","repository":{"id":103224839,"uuid":"533634056","full_name":"netobserv/pano-research","owner":"netobserv","description":null,"archived":false,"fork":false,"pushed_at":"2023-09-29T13:49:03.000Z","size":24520,"stargazers_count":1,"open_issues_count":0,"forks_count":2,"subscribers_count":6,"default_branch":"main","last_synced_at":"2025-01-24T01:39:04.161Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Zeek","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/netobserv.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null}},"created_at":"2022-09-07T06:21:48.000Z","updated_at":"2023-08-24T15:02:35.000Z","dependencies_parsed_at":null,"dependency_job_id":"98c47bae-0897-4b75-8b78-b5b15fe883a0","html_url":"https://github.com/netobserv/pano-research","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/netobserv%2Fpano-research","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/netobserv%2Fpano-research/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/netobserv%2Fpano-research/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/netobserv%2Fpano-research/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/netobserv","download_url":"https://codeload.github.com/netobserv/pano-research/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":244056431,"owners_count":20390720,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-23T22:14:29.117Z","updated_at":"2026-01-04T00:12:19.158Z","avatar_url":"https://github.com/netobserv.png","language":"Zeek","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Payload Aware Network Observability (PANO)\n\nWelcome to the project site for Payload Aware Network Observability (PANO).\n\nThe goal of this project is to provide observability to networking levels 5, 6, and 7.  Typical \nobservability tools only look at layers 3 and 4, outputing flow logs, which\nsummarize traffic flow by 5-tuple.  A great \ndeal of information can be captured looking at higher level protocols,\nsuch as DNS, HTTP, LDAP, SSL/TLS, SIP, etc.  As an \nexample of PANO's utility, we present a demo using DNS.\n\nWhile several approaches to this problem are possible, we sought to take a\nbalanced one, in terms of trading off efficiency vs. generality.\nFor example, while we could do everything in kernel space with eBPF, that\nwould require constant changes to the kernel.  Similarly, using python\nfor everything would be fast and simple, but slow due to the generality.\nInstead, we strike a middle ground: we use eBPF in the kernel for packet filtering and capture, \nfor performance reasons, and C++ or Go in user-space for efficient flexibility.\nWhere possible, where it makes sense, we re-use existing industry-standard tools.\n\n## Background\n\nAs an observability tool, PANO consists of a pipeline of service components.  The pipeline \ncaptures raw packets, analyzes them, outputs metrics, and displays them on a dashboard.  A \nmajor difference with other tools is that it does not depend on the support of a\nspecific server (e.g., a specific DNS server such as [coredns](https://coredns.io/)).  It \nuses raw packets captured on the network to reconstruct the behavior of the server.\n\nMore detail can be found in a PPT deck [here](Pano-Web-Site.pptx).\n\n## Pipeline Components\n\n- RedHat's [NetObserv eBPF Agent](https://github.com/netobserv/netobserv-ebpf-agent) -- to capture relevant packets\n- The [Zeek Network Security Monitoring Tool](https://zeek.org/) -- to analyze the relevant packets and emit the resulting logs\n- Apache [Kafka](https://kafka.apache.org/) -- to communicate between components\n- RedHat's [Flowlogs Pipeline](https://github.com/netobserv/flowlogs-pipeline) -- to convert logs to metrics\n- CNCF's [Prometheus](https://prometheus.io/) -- to scrape metrics and centrally store them\n- Grafana Labs [Grafana](https://grafana.com/) -- to visualize and display the metrics via a dashboard\n\n## Changes We Made\n\nChanges needed to the components consisted of the following:\n\n- Code Changes:\n    - [NetObserv eBPF Agent](https://github.com/netobserv/netobserv-ebpf-agent): Extensions to allow Full Packet Capture using eBPF, in addition to flow-logs.  The changes are currently in a [pull request](https://github.com/netobserv/netobserv-ebpf-agent/pull/113).  They are scheduled for inclusion in Netobserv 1.6.\n- Package Additions:\n    - An [Open-Soure Zeek Package](https://github.com/emnahum/zeek-pcapovertcp-plugin) that provides the [Zeek Network Security Monitoring Tool](https://zeek.org/) with [PCAP-over-TCP functionality](https://www.netresec.com/?page=Blog\u0026month=2022-08\u0026post=What-is-PCAP-over-IP).  Zeek itself requires no code changes. \n- Configuration (but no code changes):\n    - [Flowlogs Pipeline](https://github.com/netobserv/flowlogs-pipeline) to handle DNS logs. Config file available [here](demo1/flowlogs-pipeline/pano-kafka-dns.yaml).\n    - [Grafana](https://grafana.com/) to add a DNS dashboard. Dashboard available [here](demo1/grafana/pano-dns-dashboard.json). \n    - CNCF's [Prometheus](https://prometheus.io/) -- Minimal configuration.  Config file available [here](demo1/prometheus/prometheus.yml).\n- No Changes at all:\n    - Apache [Kafka](https://kafka.apache.org/) \n\n## Use Case: DNS\n\nTo illustrate PANO, we demonstrate a use case with DNS.  We configure the [NetObserv eBPF Agent](https://github.com/netobserv/netobserv-ebpf-agent) to capture all packets on UDP Port 53, the default DNS port.  \n\n## Workload\n\nIn addition to our pipeline, we use two instances of a DNS load generation tool, DNS-OARC's [dnsperf](https://github.com/DNS-OARC/dnsperf).  One \ninstance queries 8.8.8.8 (Google's DNS server) and another instance queries 9.0.0.1 (IBM's DNS server) simultaneously, using a sample DNS \n[querylist](demo1/dnsperf/dns-entry-list) from a trace taken at yorktown.ibm.com.\n\n## Running the Demo\n\nWe have a demo of PANO's capabilities running in Docker. \n\nA recording of the demo is available [here](pano-demo.mp4).\n\n### Requirements\n\nYou will need the following to run the demo:\n\n- A Linux bare-metal machine or VM (we tested on `Ubuntu 22.04.3 LTS`).\n- A relatively new Linux Kernel that supports CAP_BPF in Docker (tested on `5.15.0-78-generic`).\n- Git - to check out the source tree (should be included in your Linux).\n- Bash - to build the docker images from standard components (should be included in your Linux).\n- Docker and docker-compose (or \"docker compose\") that support \"host\" networking (for Ubuntu get them [here](https://docs.docker.com/engine/install/ubuntu/) and \n[here](https://docs.docker.com/compose/install/linux/)).\n\n### Check-out the Demo\n\nRun \n\n```\ngit clone git@github.com:netobserv/pano-research.git\n```\n\nto check out the source tree.\n\n### Building the Demo\n\nRun `build-me.sh` in the demo1 subdirectory:\n\n```\nbuild-me.sh\n```\n\nYou should see various component images downloading and some building.\n\n### Running the Demo\n\nIn the demo1 subdirectory: \n\nRun `docker compose up -d` \n\n```\ndocker compose up -d\n```\nOR `docker-compose up -d` \n```\ndocker-compose up -d\n```\ndepending on which version of `docker` and `docker-compose` you have.\n\nYou should see the various components come up:\n```\nCreating netobserv-ebpf-agent ... done\nCreating zeek                 ... done\nCreating kafka                ... done\nCreating flowlogs-pipeline    ... done\nCreating prometheus           ... done\nCreating grafana              ... done\nCreating dnsperf1             ... done\nCreating dnsperf2             ... done\n```\n\nNow in a browser window, go to:\n\n```\nhttp://pano.sl.cloud9.ibm.com:3000/\n```\n\n(if you are running on a different machine, substitute `your-machine-name` for `pano.sl.cloud9.ibm.com`)\n\nThe first time you do this, you will see the following:\n\n![Grafana Login Page](images/grafana-foobar.png)\n\nGrafana is asking you to log in.  The credentials are `admin` and `admin`.  If you change them, you must remember them for next time.\nOur advice is to **not** change them and **skip** the password change.  \n\nYou may be presented with a `Dashboard` page.  If so, click the `Home` button in the upper left.\n\nYou should see a Welcome page:\n\n![Grafana Welcome Page](images/grafana-welcome.png)\n\nIn the lower left corner, click on the `PANO DNS Metrics`.\n\nYou should see something like the following:\n\n![Grafana DNS Page](images/grafana-pano.png)\n\nAt first, the page will not display anything, but in about 10-15 seconds it should start to populate.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnetobserv%2Fpano-research","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fnetobserv%2Fpano-research","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fnetobserv%2Fpano-research/lists"}