{"id":13821377,"url":"https://github.com/flarco/elk-tcpdump","last_synced_at":"2026-01-16T16:49:57.184Z","repository":{"id":132725124,"uuid":"67744079","full_name":"flarco/elk-tcpdump","owner":"flarco","description":"Code to aggregate tcpdump traffic and send to ELK (Elasticsearch-Logstach-Kibana)","archived":false,"fork":false,"pushed_at":"2016-09-09T00:19:27.000Z","size":12,"stargazers_count":18,"open_issues_count":0,"forks_count":7,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-11-19T21:36:03.760Z","etag":null,"topics":["elasticsearch","elasticsearch-logstach-kibana","elk","tcpdump"],"latest_commit_sha":null,"homepage":null,"language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/flarco.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2016-09-08T22:14:33.000Z","updated_at":"2024-11-04T02:31:44.000Z","dependencies_parsed_at":null,"dependency_job_id":"43312edb-29a3-48f1-97d9-3416f40826ab","html_url":"https://github.com/flarco/elk-tcpdump","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flarco%2Felk-tcpdump","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flarco%2Felk-tcpdump/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flarco%2Felk-tcpdump/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/flarco%2Felk-tcpdump/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/flarco","download_url":"https://codeload.github.com/flarco/elk-tcpdump/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254530647,"owners_count":22086652,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["elasticsearch","elasticsearch-logstach-kibana","elk","tcpdump"],"created_at":"2024-08-04T08:01:20.981Z","updated_at":"2026-01-16T16:49:57.167Z","avatar_url":"https://github.com/flarco.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# elk-tcpdump\nCode to aggregate tcpdump traffic and send to ELK (Elasticsearch-Logstach-Kibana)\n\nThis allows one to capture a host's network traffic statistics:\n  - Source IP/Host/Port to Target IP/Host/Port\n  - Aggregate count of packets over time\n  - Aggregate length of packets over time\n\nThis is ideal to run on firewalls (such as PfSense) to monitor traffic with a service such as ELK (\u003chttps://www.elastic.co\u003e)\n\n# Instructions\n\nThis was only tested on Ubuntu 14.04 and FreeBSD. The following packages are required:\n- tcpdump\n- host\n\nThen clone this repo:\n```\ngit clone https://github.com/flarco/elk-tcpdump.git\ncd elk-tcpdump\n```\n\nTo start collecting tcpdump statistics, run the following on the host:\n```shell\ntcpdump -U -i eth0 -nn -tttt port not 5141 | python tcpdump_aggregate.py \"192.168.2.3:5141\"\n\n# this is an example on a firewall with 2 NICs\ntcpdump -U -i eth0 -i eth1 -nn -tttt port not 5141 | python tcpdump_aggregate.py \"192.168.2.3:5141\"\n ```\n\nIn the example above, the tcpdump aggregates will be sent over to host '192.168.2.3' / port 5141 via UDP every interval specified (in the script - default 20 sec).\n\nHere is an example of the received data on 192.168.2.3:5141:\n```shell\nroot@d54ea1457852:/tmp# netcat -ul 5141\n{\"source_IP\": \"172.17.0.3\", \"source_PORT\": 22, \"target_IP\": \"172.17.0.1\", \"target_PORT\": 54686, \"type\": \"TCP\", \"count\": 1, \"length\": 212, \"source_HOST\": \"172.17.0.3\", \"target_HOST\": \"172.17.0.1\", \"time\": \"2016-09-08 23:27:40.090202\"}\n{\"source_IP\": \"172.17.0.1\", \"source_PORT\": 54692, \"target_IP\": \"172.17.0.3\", \"target_PORT\": 22, \"type\": \"TCP\", \"count\": 24, \"length\": 0, \"source_HOST\": \"NXDOMAIN\", \"target_HOST\": \"NXDOMAIN\", \"time\": \"2016-09-08 23:28:29.073292\"}\n{\"source_IP\": \"172.17.0.1\", \"source_PORT\": 54690, \"target_IP\": \"172.17.0.3\", \"target_PORT\": 22, \"type\": \"TCP\", \"count\": 1, \"length\": 52, \"source_HOST\": \"172.17.0.1\", \"target_HOST\": \"172.17.0.3\", \"time\": \"2016-09-08 23:28:29.073292\"}\n{\"source_IP\": \"172.17.0.3\", \"source_PORT\": 22, \"target_IP\": \"172.17.0.1\", \"target_PORT\": 54690, \"type\": \"TCP\", \"count\": 1, \"length\": 0, \"source_HOST\": \"172.17.0.3\", \"target_HOST\": \"172.17.0.1\", \"time\": \"2016-09-08 23:28:29.073292\"}\n{\"source_IP\": \"172.17.0.3\", \"source_PORT\": 22, \"target_IP\": \"172.17.0.1\", \"target_PORT\": 54692, \"type\": \"TCP\", \"count\": 24, \"length\": 3888, \"source_HOST\": \"172.17.0.3\", \"target_HOST\": \"172.17.0.1\", \"time\": \"2016-09-08 23:28:29.073292\"}\n{\"source_IP\": \"172.17.0.1\", \"source_PORT\": 54686, \"target_IP\": \"172.17.0.3\", \"target_PORT\": 22, \"type\": \"TCP\", \"count\": 1, \"length\": 0, \"source_HOST\": \"172.17.0.1\", \"target_HOST\": \"172.17.0.3\", \"time\": \"2016-09-08 23:28:29.073292\"}\n```\n\nThe `source_HOST` / `target_HOST` fields are resolved by trial (using the linux `host` command), then cached. If no hostname is returned, the IP is stored instead.\n\nWith this process, we can use Logstash to parse this data and ingest into Elasticsearch, then view in Kibana.\n\nWe can use the `logstash.conf` file for this. Make sure to [create the index-template](https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-templates.html) prior to ingesting in Elasticsearch!\n\n------\n\nHere is an example of Kibana using this data:\n\n![image](https://cloud.githubusercontent.com/assets/7671010/18371213/895bf418-7600-11e6-846d-a70a67efdd13.png)","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflarco%2Felk-tcpdump","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fflarco%2Felk-tcpdump","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fflarco%2Felk-tcpdump/lists"}