{"id":13585439,"url":"https://github.com/faucetsdn/poseidon","last_synced_at":"2025-12-13T22:09:42.938Z","repository":{"id":37038304,"uuid":"58469159","full_name":"faucetsdn/poseidon","owner":"faucetsdn","description":"Poseidon is a python-based application that leverages software defined networks (SDN) to acquire and then feed network traffic to a number of machine learning techniques.  The machine learning algorithms classify and predict the type of device.","archived":false,"fork":false,"pushed_at":"2025-03-26T23:01:44.000Z","size":21074,"stargazers_count":430,"open_issues_count":0,"forks_count":127,"subscribers_count":24,"default_branch":"main","last_synced_at":"2025-05-15T20:14:26.870Z","etag":null,"topics":["automation","docker","faucet","hacktoberfest","machine-learning","network-analysis","network-forensics","network-monitoring","network-traffic","network-traffic-capture","network-traffic-classification","networking","pcap","pcap-analyzer","pcap-files","sdn","sdn-controller","security","software-defined-network"],"latest_commit_sha":null,"homepage":"","language":"Python","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/faucetsdn.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":"CONTRIBUTING.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":"AUTHORS","dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2016-05-10T14:40:30.000Z","updated_at":"2025-04-08T04:45:06.000Z","dependencies_parsed_at":"2023-08-29T03:31:18.273Z","dependency_job_id":"71b78c06-b6c7-4403-9b74-e1c86c31a1cd","html_url":"https://github.com/faucetsdn/poseidon","commit_stats":null,"previous_names":["faucetsdn/poseidon","iqtlabs/poseidon"],"tags_count":106,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/faucetsdn%2Fposeidon","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/faucetsdn%2Fposeidon/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/faucetsdn%2Fposeidon/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/faucetsdn%2Fposeidon/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/faucetsdn","download_url":"https://codeload.github.com/faucetsdn/poseidon/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254414372,"owners_count":22067275,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["automation","docker","faucet","hacktoberfest","machine-learning","network-analysis","network-forensics","network-monitoring","network-traffic","network-traffic-capture","network-traffic-classification","networking","pcap","pcap-analyzer","pcap-files","sdn","sdn-controller","security","software-defined-network"],"created_at":"2024-08-01T15:04:56.523Z","updated_at":"2025-12-13T22:09:37.287Z","avatar_url":"https://github.com/faucetsdn.png","language":"Python","funding_links":[],"categories":["Python"],"sub_categories":[],"readme":"# Poseidon\n\n[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n![Build Status](https://github.com/faucetsdn/poseidon/workflows/test/badge.svg)\n[![codecov](https://codecov.io/github/faucetsdn/poseidon/graph/badge.svg?token=PJmRKQMk1P)](https://codecov.io/github/faucetsdn/poseidon)\n[![Docker Hub Downloads](https://img.shields.io/docker/pulls/iqtlabs/poseidon.svg)](https://hub.docker.com/r/iqtlabs/poseidon/)\n\n\u003e Software Defined Network Situational Awareness\n\n\u003cimg src=\"/docs/img/poseidon-logo.png\" width=\"67\" height=\"93\" hspace=\"20\"/\u003e\u003ca href=\"https://web.archive.org/web/20170316012151/https://www.blackducksoftware.com/open-source-rookies-2016\" \u003e\u003cimg src=\"/docs/img/Rookies16Badge_1.png\" width=\"100\" alt=\"POSEIDON is now BlackDuck 2016 OpenSource Rookie of the year\" hspace=\"20\"\u003e\u003c/a\u003e\n\nPoseidon began as a joint effort between two of the IQT Labs: [Cyber Reboot](https://www.cyberreboot.org/) and [Lab41](http://www.lab41.org/). The project's goal is to explore approaches to better identify what nodes are on a given (computer) network and understand what they are doing.  The project utilizes Software Defined Networking and machine learning to automatically capture network traffic, extract relevant features from that traffic, perform classifications through trained models, convey results, and provide mechanisms to take further action. While the project works best leveraging modern SDNs, parts of it can still be used with little more than packet capture (pcap) files.\n\n## Table of Contents\n\n- [Background](#background)\n- [Prerequisites](#prerequisites)\n- [Installing or updating Poseidon](#installing)\n- [SDN Controller Configuration](#sdn-controller-configuration)\n    - [Faucet Configuration](#faucet-configuration)\n- [Usage](#usage)\n- [Troubleshooting](#troubleshooting)\n- [Network Data Logging](#logging)\n- [Related Components](#related-components)\n- [Additional Info](#additional-info)\n\n## Background\n\nThe Poseidon project originally began as an experiment to test the merits of leveraging SDN and machine learning techniques to detect abnormal network behavior. (Please read our [blogs posts](#additional-info) linked below for several years of background) While that long-term goal remains, the unfortunate reality is that the state of rich, labelled, public, and MODERN network data sets for ML training is pretty poor. Our lab is working on improving the availability of network training sets, but in the near term the project remains focused on 1) improving the accuracy of identifying what a node *IS* (based on captured IP header data) and 2) developing Poseidon into a \"harness\" of sorts to house machine learning techniques for additional use cases. (Read: Not just ours!)\n\n\n## Prerequisites\n\n- [Docker](https://www.docker.com/) - Poseidon and related components run on top of Docker, so understanding the fundamentals will be useful for troubleshooting as well.  Note: installing via Snap is currently unsupported. [A Good Ubuntu Docker Quick-Start](https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-on-ubuntu-16-04)\n- [Compose](https://docs.docker.com/compose/) - Poseidon is orchestrated with [docker-compose](https://docs.docker.com/compose/install/). You will need a version that supports compose file format version 3 and health check conditions (minimum 1.29.2).\n- [Curl](https://curl.haxx.se/download.html) - command-line for transferring data with URLs.\n- [git](https://git-scm.com/downloads) - distributed version control system.\n- [jq](https://stedolan.github.io/jq/download/) - command-line JSON processor.\n- An SDN Controller - specifically [Faucet](https://faucet.nz/)\n- ~10GB of free disk space\n\n\u003e Note: Installation on `OS X` is possible but not supported.\n\n## Installing\n\n### Permissions for Docker\n\nTo simplify using commands with Docker, we recommend allowing the user that will be executing Poseidon commands be part of the `docker` group so they can execute Docker commands without `sudo`.  Typically, this can be done with:\n```\nsudo usermod -aG docker $USER\n```\nFollowed by closing the existing shell and starting a new one.\n\n### Getting the bits\n\n*NOTE: If you have previously installed Poseidon from a .deb package, please remove it first. Installation from .deb is no longer supported.*\n\nInstall the poseidon script which we will use to install and manage Poseidon.\n\n```\ncurl -L https://raw.githubusercontent.com/IQTLabs/poseidon/main/bin/poseidon -o /usr/local/bin/poseidon\nchmod +x /usr/local/bin/poseidon\n```\n\n#### Faucet Configuration\n\u003cimg src=\"/docs/img/faucet.png\" width=\"190\" height=\"100\"\u003e\nNOTE: Poseidon requires at least Faucet version 1.9.46 or higher.\n\nPoseidon uses a faucetconfrpc server, to maintain Faucet configuration. Poseidon starts its own server for you by default, and also by default Poseidon and Faucet have to be on the same machine. To run Faucet on a separate machine, you will need to start faucetconfrpc on that other machine, and update `faucetconfrpc_address` to point to where the faucetconfrpc is running. You may also need to update `faucetconfrpc_client`, if you are not using the provided automatically generated keys.\n\nIf you have Faucet running already, make sure Faucet is started with the following environment variables, which allow Poseidon to change its config, and receive Faucet events:\n\n```\nexport FAUCET_EVENT_SOCK=1\nexport FAUCET_CONFIG_STAT_RELOAD=1\n```\n\nFaucet is now configured and ready for use with Poseidon.\n\n##### Faucet stacking\n\nFaucet supports stacking (distributed switching - multiple switches acting together as one).  Poseidon also supports this - Poseidon's mirroring interface should be connected to a port on the root switch.  You will need to allocate a port on each non-root switch also, and install a loopback plug (either Ethernet or fiber) in that port.  Poseidon will detect stacking and take care of the rest of the details (using Faucet's tunneling feature to move mirror packets from the non-root switches to the root switch's mirror port).  The only Poseidon config required is to add the dedicated port on each switch to the `controller_mirror_port` dictionary.\n\n## Configuring Poseidon\n\nYou will need to create a directory and config file on the server where Poseidon will run.\n\n```\nsudo mkdir /opt/poseidon\nsudo cp config/poseidon.config /opt/poseidon\n```\n\nNow, edit this file. You will need to set at minimum:\n\n* controller_type, as appropriate to the controller you are running (see above).\n* collector_nic: must be set to the interface name on the server, that is connected to the switch mirror port.\n* controller_mirror_ports: must be set to the interface on the switch that will be used as the mirror port.\n\nOptionally, you may also set controller_proxy_mirror_ports (for switches that don't have their own mirror ports, and can be mirrored with another switch).\n\n\n## Updating Poseidon\n\nFrom v0.10.0, you can update an existing Poseidon installation with `poseidon -u` (your configuration will be preserved). Updating from previous versions is not supported - please remove and reinstall as above. You can also give `poseidon -u` a specific git hash if you want to update to an unreleased version.\n\n\n## Usage\n\nAfter installation you'll have a new command `poseidon` available for looking at the configuration, logs, and shell, as well as stopping and starting the service.\n```\n$ poseidon help\nPoseidon, an application that leverages software defined networks (SDN) to acquire and then feed network traffic to a number of machine learning techniques. For more info visit: https://github.com/IQTLabs/poseidon\n\nUsage: poseidon [option]\nOptions:\n    -a,  api           get url to the Poseidon API\n    -c,  config        display current configuration info\n    -d,  delete        delete Poseidon installation (uses sudo)\n    -e,  shell         enter into the Poseidon shell, requires Poseidon to already be running\n    -h,  help          print this help\n    -i,  install       install Poseidon repo (uses sudo)\n    -l,  logs          display the information logs about what Poseidon is doing\n    -r,  restart       restart the Poseidon service (uses sudo)\n    -s,  start         start the Poseidon service (uses sudo)\n    -S,  stop          stop the Poseidon service (uses sudo)\n    -u,  update        update Poseidon repo, optionally supply a version (uses sudo)\n    -V,  version       get the version installed\n```\n\nStep 0:\n\nOptionally specify a prefix location to install Poseidon by setting an environment variable, if it is unset, it will default to `/opt` and Poseidon. (If using Faucet, it will also override `/etc` locations to this prefix.)\n\n```\nexport POSEIDON_PREFIX=/tmp\n```\n\nStep 1:\n\n```\nposeidon install\n```\n\nStep 2:\n\nConfigure Poseidon for your preferred settings. Open `/opt/poseidon/poseidon.config` (add the Poseidon prefix if you specified one).\n\nFor using Faucet, make sure to minimally change the `controller_mirror_ports` to match the switch name and port number of your mirror port.  You will also need to update the `collector_nic` in the `poseidon` section to match the interface name of the NIC your mirror port is connected to.\n\nStep 3:\n\nIf you don't have Faucet already and/or you want to Poseidon to spin up Faucet for you as well, simply run the following command and you will be done:\n\n```\nposeidon start\n```\n\nStep 4:\n\nIf you are using your own installation of Faucet, you will need to enable communication between Poseidon and Faucet. Poseidon needs to change Faucet's configuration, and Faucet needs to send events to Poseidon. This configuration needs to be set with environment variables (see https://docs.faucet.nz/). For example, if running Faucet with Docker, you will need the following environment configuration in the `faucet` service in your docker-compose file:\n\n```\n        environment:\n            FAUCET_CONFIG: '/etc/faucet/faucet.yaml'\n            FAUCET_EVENT_SOCK: '/var/run/faucet/faucet.sock'\n            FAUCET_CONFIG_STAT_RELOAD: '1'\n```\n\nIf Faucet and Poseidon are running on the same machine, you can start Poseidon and you will be done:\n\n```\nposeidon start --standalone\n```\n\nStep 5:\n\nIf you are running Faucet and Poseidon on different machines, configuration is more complex (work to make this easier is ongoing): execute Step 4 first. Then you will need to run `event-adapter-rabbitmq` and `faucetconfrpc` services on the Faucet host, and change Poseidon's configuration to match.\n\nFirst start all services from `helpers/faucet/docker-compose.yaml` on the Faucet host, using a Docker network that has network connectivity with your Poseidon host. Set `FA_RABBIT_HOST` to be the address of your Poseidon host. `faucet_certstrap` will generate keys in `/opt/faucetconfrpc` which will need to be copied to your Poseidon host. Then modify `faucetconfrpc_address` in `/opt/poseidon/config/poseidon.config` to point to your Faucet host.\n\nYou can now start Poseidon:\n\n```\nposeidon start --standalone\n```\n\n## Troubleshooting\n\nPoseidon by its nature depends on other systems. The following are some common issues and troubleshooting steps.\n\n### Poseidon doesn't detect any hosts.\n\nThe most common cause of this problem, with the FAUCET controller, is RabbitMQ connectivity.\n\n* Check that the RabbitMQ event adapter (faucet/event-adapter-rabbitmq) is running and not restarting.\n\n```\n# docker ps|grep faucet/event-adapter-rabbitmq\n4a7509829be0        faucet/event-adapter-rabbitmq           \"/usr/local/bin/entr…\"   3 days ago          Up 3 days\n```\n\n* Check that FAUCET.Event messages are being received by Poseidon.\n\nThis command reports the time that the most recent FAUCET.Event message was received by Poseidon.\n\nIf run repeatedly over a couple of minutes this timestamp should increase.\n\n```\ndocker exec -it poseidon_poseidon_1 /bin/sh\n/poseidon # wget -q -O- localhost:9304|grep -E ^poseidon_last_rabbitmq_routing_key_time.+FAUCET.Event\nposeidon_last_rabbitmq_routing_key_time{routing_key=\"FAUCET.Event\"} 1.5739482267393966e+09\n/poseidon # wget -q -O- localhost:9304|grep -E ^poseidon_last_rabbitmq_routing_key_time.+FAUCET.Event\nposeidon_last_rabbitmq_routing_key_time{routing_key=\"FAUCET.Event\"} 1.5739487978768678e+09\n/poseidon # exit\n```\n\n### Poseidon doesn't report any host roles.\n\n* Check that the mirror interface is up and receiving packets (should be configured in `collector_nic`. The interface must be up before Posiedon starts.\n\n```\n# ifconfig enx0023559c2781\nenx0023559c2781: flags=4163\u003cUP,BROADCAST,RUNNING,MULTICAST\u003e  mtu 1500\n        inet6 fe80::223:55ff:fe9c:2781  prefixlen 64  scopeid 0x20\u003clink\u003e\n        ether 00:23:55:9c:27:81  txqueuelen 1000  (Ethernet)\n        RX packets 82979981  bytes 77510139268 (77.5 GB)\n        RX errors 0  dropped 0  overruns 0  frame 0\n        TX packets 202  bytes 15932 (15.9 KB)\n        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0\n```\n\n* Check that there is disk space available and pcaps are being accumulated in */opt/poseidon_files* (add `POSEIDON_PREFIX` in front if it was used.)\n\n```\n# find /opt/poseidon_files -type f -name \\*pcap |head -5\n/opt/poseidon_files/trace_d3f3217106acd75fe7b5c7069a84a227c9e48377_2019-11-15_03_10_41.pcap\n/opt/poseidon_files/tcprewrite-dot1q-2019-11-15-06_26_48.529473-UTC/pcap-node-splitter-2019-11-15-06_26_50.192570-UTC/clients/trace_0a6ce9490c193b65c3cad51fffbadeaed4ed5fdd_2019-11-15_06_11_24-client-ip-216-58-196-147-192-168-254-254-216-58-196-147-vssmonitoring-frame-eth-ip-icmp.pcap\n/opt/poseidon_files/tcprewrite-dot1q-2019-11-15-06_26_48.529473-UTC/pcap-node-splitter-2019-11-15-06_26_50.192570-UTC/clients/trace_0a6ce9490c193b65c3cad51fffbadeaed4ed5fdd_2019-11-15_06_11_24-miscellaneous-192-168-254-1-192-168-254-254-vssmonitoring-frame-eth-arp.pcap\n/opt/poseidon_files/tcprewrite-dot1q-2019-11-15-06_26_48.529473-UTC/pcap-node-splitter-2019-11-15-06_26_50.192570-UTC/clients/trace_0a6ce9490c193b65c3cad51fffbadeaed4ed5fdd_2019-11-15_06_11_24-client-ip-192-168-254-254-192-168-254-254-74-125-200-189-udp-frame-eth-ip-wsshort-port-443.pcap\n/opt/poseidon_files/tcprewrite-dot1q-2019-11-15-06_26_48.529473-UTC/pcap-node-splitter-2019-11-15-06_26_50.192570-UTC/servers/trace_0a6ce9490c193b65c3cad51fffbadeaed4ed5fdd_2019-11-15_06_11_24-server-ip-74-125-68-188-192-168-254-254-74-125-68-188-frame-eth-ip-tcp-port-5228.pcap\n```\n\n\n## Developing\n\n### Modifying Code that Runs in a Docker Container\n\nIf installed as described above, poseidon's codebase will be at `/opt/poseidon`.  At this location, make changes, then run `poseidon restart`.\n\n## Network Data Logging\n\nPoseidon logs some data about the network it monitors. Therefore it is important to secure Poseidon's own host (aside from logging, Poseidon can of course change FAUCET's network configuration).\n\nThere are two main types of logging at the lowest level. The first is FAUCET events - FAUCET generates an event when it learns on which port a host is present on the network, and the event includes source and destination Ethernet MAC and IP addresses (if present). For example:\n\n```\n2019-11-21 20:18:41,909 [DEBUG] faucet - got faucet message for l2_learn: {'version': 1, 'time': 1574367516.3555572, 'dp_id': 1, 'dp_name': 'x930', 'event_id': 172760, 'L2_LEARN': {'port_no': 22, 'previous_port_no': None, 'vid': 254, 'eth_src': '0e:00:00:00:00:99', 'eth_dst': '0e:00:00:00:00:01', 'eth_type': 2048, 'l3_src_ip': '192.168.254.3', 'l3_dst_ip': '192.168.254.254'}}\n```\n\nThe second type of logging is host based pcap captures, with most of the application (L4) payload removed. Poseidon causes the `ncapture` component (https://github.com/IQTLabs/network-tools/tree/main/network_tap/ncapture) to capture traffic, which is logged in `/opt/poseidon_files`. These are used in turn to learn host roles, etc.\n\n\n## Related Components\n\n- [NetworkML](https://github.com/IQTLabs/NetworkML)\n- [network-tools](https://github.com/IQTLabs/network-tools)\n\n## Additional Info\n\n- [Authors](AUTHORS)\n- Blog posts:\n  - [How to Install Poseidon and get it working with Faucet SDN](https://blog.cyberreboot.org/how-to-install-poseidon-and-get-it-working-with-faucet-sdn-c3cdeed1901f)\n  - [Running Poseidon on a 100G Netowork](https://blog.cyberreboot.org/running-poseidon-on-a-100g-network-8def4dc8eecd)\n  - [Using machine learning to classify devices on your network](https://blog.cyberreboot.org/using-machine-learning-to-classify-devices-on-your-network-e9bb98cbfdb6)\n  - [CRviz: Initial Release](https://blog.cyberreboot.org/crviz-initial-release-45a3023e0e93)\n  - [CRviz: Scalable design for network visualization](https://blog.cyberreboot.org/crviz-scalable-design-for-network-visualization-14689133fd91)\n  - [A better way to visualize what’s on our networks?](https://blog.cyberreboot.org/a-better-way-to-visualize-whats-on-our-networks-4f87fd42da6)\n  - [TCPDump, and the care and feeding of an intelligent SDN](https://blog.cyberreboot.org/tcpdump-and-the-care-and-feeding-of-an-intelligent-sdn-eca6e7506342)\n  - [The Case for Detecting Lateral Movement](https://blog.cyberreboot.org/the-case-for-detecting-lateral-movement-2018ae631b04)\n  - [Poseidon with FAUCET SDN Controller](https://blog.cyberreboot.org/poseidon-with-faucet-sdn-controller-b5e78e46660b)\n  - [Building a Software-Defined Network with Raspberry Pis and a Zodiac FX switch](https://blog.cyberreboot.org/building-a-software-defined-network-with-raspberry-pis-and-a-zodiac-fx-switch-97184032cdc1)\n  - [Thanks to FAUCET, Poseidon Now Supports Switches Running OpenFlow 1.3](https://blog.cyberreboot.org/thanks-to-faucet-poseidon-now-supports-switches-running-openflow-1-3-e5489f2bc1f5)\n  - [Deep Session Learning for Cyber Security](https://blog.cyberreboot.org/deep-session-learning-for-cyber-security-e7c0f6804b81)\n  - [Introducing Vent](https://blog.cyberreboot.org/introducing-vent-1d883727b624)\n  - [SDN and the need for more (security) verbs](https://blog.cyberreboot.org/sdn-and-the-need-for-more-security-verbs-a6315935fca4)\n- See the latest changes [here](CHANGELOG.md).\n- [Code of Conduct](CODE_OF_CONDUCT.md)\n- Want to contribute? Awesome! Issue a pull request or see more details [here](CONTRIBUTING.md).\n- Developer Guide\n- [License](LICENSE)\n- [Maintainers](MAINTAINERS)\n- [Releases](https://github.com/IQTLabs/poseidon/releases)\n- [Tests](https://github.com/IQTLabs/poseidon/actions)\n- [Version](VERSION)\n- Videos:\n  - [Installing Poseidon and Faucet together](https://www.youtube.com/watch?v=Qst5oNs5uY0)\n  - [Analyzing and Visualizing Networks: The Powerful Combination of Poseidon and CRviz](https://www.youtube.com/watch?v=mTe5ajQRNL0)\n  - [Poseidon + Faucet SDN](https://www.youtube.com/watch?v=nOgWze_4HOU)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffaucetsdn%2Fposeidon","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ffaucetsdn%2Fposeidon","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ffaucetsdn%2Fposeidon/lists"}