{"id":13588782,"url":"https://github.com/AlexanderLevenskikh/network-fingerprinter","last_synced_at":"2025-04-08T06:32:46.055Z","repository":{"id":41752709,"uuid":"217919261","full_name":"AlexanderLevenskikh/network-fingerprinter","owner":"AlexanderLevenskikh","description":"Purely passive OS and application fingerprinting system","archived":false,"fork":false,"pushed_at":"2023-01-24T00:51:22.000Z","size":4282,"stargazers_count":23,"open_issues_count":46,"forks_count":4,"subscribers_count":0,"default_branch":"master","last_synced_at":"2024-11-06T08:43:15.277Z","etag":null,"topics":["application-detection","elasticsearch-client","fingerprint","fingerprint-scanner","fingerprint-sensor","fingerprints","network-fingerprinter","os-detection","p0f","passive-os-fingerprint","pcap-analyzer","signature","signature-analysis","traffic","traffic-analysis","traffic-analyzer","tshark","wireshark"],"latest_commit_sha":null,"homepage":"","language":"TypeScript","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/AlexanderLevenskikh.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2019-10-27T21:30:58.000Z","updated_at":"2024-04-02T01:09:50.000Z","dependencies_parsed_at":"2023-02-13T06:45:35.799Z","dependency_job_id":null,"html_url":"https://github.com/AlexanderLevenskikh/network-fingerprinter","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlexanderLevenskikh%2Fnetwork-fingerprinter","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlexanderLevenskikh%2Fnetwork-fingerprinter/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlexanderLevenskikh%2Fnetwork-fingerprinter/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/AlexanderLevenskikh%2Fnetwork-fingerprinter/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/AlexanderLevenskikh","download_url":"https://codeload.github.com/AlexanderLevenskikh/network-fingerprinter/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":247792178,"owners_count":20996878,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["application-detection","elasticsearch-client","fingerprint","fingerprint-scanner","fingerprint-sensor","fingerprints","network-fingerprinter","os-detection","p0f","passive-os-fingerprint","pcap-analyzer","signature","signature-analysis","traffic","traffic-analysis","traffic-analyzer","tshark","wireshark"],"created_at":"2024-08-01T15:06:55.548Z","updated_at":"2025-04-08T06:32:41.042Z","avatar_url":"https://github.com/AlexanderLevenskikh.png","language":"TypeScript","funding_links":[],"categories":["TypeScript"],"sub_categories":[],"readme":"# Network fingerprinter\r\n\r\n\u003e Network fingerprinter is a scalable, easy to deploy network traffic inspection system for purely passive OS and application fingerprinting\r\n\r\n*Read this in other languages: [English](README.md), [Russian](README.ru.md).*\r\n\r\n## Table of Contents\r\n\r\n- [Purpose](#purpose)\r\n- [System architecture](#system-architecture)\r\n\t- [Sensor](#sensor)\r\n\t- [Collector](#collector)\r\n\t- [Analyzer](#analyzer)\r\n- [Installation](#installation)\r\n- [Signature analysis](#signature-analysis)\r\n\r\n## Purpose\r\nWell-known systems for passive OS fingerprinting such as p0f, satori, etc are powerful, but have some disadvantages for production use:\r\n* Embrassing inspection huge traffic alerts\r\n* For passive fingerprinting it's not necessary to store all traffic dump, except flow samples such as handshake tcp SYN/SYN+ACK, tls client hello and others required for signature analysis packet samples. Flows are powerful, but existing solutions allow to store either all trafic or alerts only\r\n* Lack of GUI\r\n* Complicated production configuration\r\n\r\n## System architecture\r\n\r\nInternally, system consists of 3 modules:\r\n* [Sensor](#sensor)\r\n* [Collector](#collector)\r\n* [Analyzer](#analyzer)\r\n\r\n#### Sensor \r\n\r\nSensor is a **tshark**-driven selective traffic exporter, it's exports only SYN, SYN+ACK, TLS client hello and HTTP requests/responses to external **elasticsearch** index. System can have as many sensors as needs.\r\n\r\n#### Collector\r\n\r\nCollector is an **elasticsearch** cluster. Also, system can have more than one collector if needs.\r\n\r\n#### Analyzer\r\n\r\nAnalyzer is a **kibana** GUI for **packets-\\*** index inspection and **analytics-app** traffic fingerprints view. Also, analytics-app allows to upload existing pcap[ng] dump.\r\n\r\n\u003e All system components can be installed on signle host\r\n\r\n## Installation\r\n\r\nEvery module has an `.env` file with **docker-compose** configuration, you should replace data in this file. \r\n\r\n##### Firstly, install collector module:\r\nYou needs to create folder which will be used elasticsearch for persist data\r\nIn our sample, it's a `./data` directory in `collector` folder, as configured in `docker-compose.yml`:\r\n```\r\nvolumes:\r\n  elasticsearch:\r\n    driver_opts:\r\n      type: none\r\n      device: $PWD/data\r\n      o: bind\r\n```\r\nYou need to replace string `device: $PWD/data`  and make sure that this directory exists, or remove lines `3-6` for use docker-driven volume.\r\n\r\nAll configuration steps: \r\n```\r\ncd  collector\r\nvim .env # set environment variables\r\nvim docker-compose.yml # set elasticsearch volume device \r\ndocker-compose up -d # run in detach mode\r\n```\r\nWait, until elasticsearch ready. (see docker logs CONTAINER_NAME)\r\n\r\n##### Next, let's install sensor:\r\n\r\nIf elasticsearch is ready (but it's not necessary - containers always restart on failure), simply edit `.env` in `sensor` directory and run `docker-compose up -d`\r\n\r\n##### Prepare analyzer\r\n\r\nNow, we can install and run analyzer as simple as edit `.env` and run `docker-compose up -d` in `analytics-app` directory.\r\nOpen `http://${host}:5601` to open kibana GUI and `http://${host}:3000` to open analytics app, where `host` is your ip or domain name.\r\n\r\n## Signature analysis\r\n\r\nApp consists of three tabs:\r\n* Streams\r\n* Statistics\r\n* Upload \\*.pcap\r\n\r\nFirst tab displays filtered streams list:\\\r\n![Streams list](https://i.imgur.com/6QBX5vh.png)\r\n\r\n\r\nYou can filter streams using `Search` form:\\\r\n![Filter](https://i.imgur.com/G3YFbCi.png)\r\n\r\nTo store data in buffer (for paste in filter, for example) click `copy` button beside data:\\\r\n![Copy](https://i.imgur.com/CEoOfhZ.png)\r\n\r\nColumns description:\r\n* First column consists **common data** about stream:\r\n\t* **SensorID** - for sensor it matches pattern 'sensor-\\*', for pcap-player - 'pcap-player-*'\r\n\t* **SNI** (server name indication from tls headers, if extension used)\r\n\t* **Application layer protocols** - list of all application protocols used in stream\r\n* **Date and time** - timestamp of first captured packet in TCP stream (usually, SYN, but can be http request or TLS client hello)\r\n* **MAC, IP, Port.** Source is a host, that initiates TCP request (client) and destination is a host, that receives request and send TCP response (server), but as known, TCP is full-duplex protocol and the names \"client\" and \"server\" are conditional.\r\n* **Fingerprint** - defined fingerprints of stream, they may include TCP, TLS and HTTP fingerprints for source (by SYN, HTTP request and TLS client hello) and TCP, HTTP fingerprints for destination (by SYN+ACK, HTTP response)\r\n\r\nStatistics menu item has two tabs: *by clients* and *by servers*:\\\r\n![stat-tabs](https://i.imgur.com/CoGpUNB.png)\r\n\r\nEach tab has a grid with (ip, mac) discovered tuples from traffic.\\\r\n![by-clients](https://i.imgur.com/FBhRkMb.png)\r\n\r\nEvery row is clickable and you can see detailed information about selected host by click on row (opening in right-side drawer):\\\r\n![drawer](https://i.imgur.com/JKTtCAw.png)\r\nAs you can see, this information includes all fingerprints from stream and reference to these streams list, also in this example showed case of multiple discovered fingerprints (TLS) and matches from SSL blacklist (Dridex, Tofsee). \r\n\r\nPcap uploading:\\\r\n![pcap-upload](https://i.imgur.com/dTQf184.png)\\\r\nSimply drag'n'drop captured traffic dump (one or more files) or click directly on area to upload. After processing, streams will be available on TCP streams tab, if dump includes significant for signature analysis packets (SYN, SYN+ACK, HTTP request/response or TLS client hello)\r\n\r\n## License\r\nThis project is licensed under the terms of the Apache 2.0 open source license. Please refer to [LICENSE](LICENSE) for the full terms.\r\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAlexanderLevenskikh%2Fnetwork-fingerprinter","html_url":"https://awesome.ecosyste.ms/projects/github.com%2FAlexanderLevenskikh%2Fnetwork-fingerprinter","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2FAlexanderLevenskikh%2Fnetwork-fingerprinter/lists"}