{"id":26794619,"url":"https://github.com/rusticata/pcap-analyzer","last_synced_at":"2025-07-30T00:33:24.360Z","repository":{"id":40354505,"uuid":"193905076","full_name":"rusticata/pcap-analyzer","owner":"rusticata","description":"PAL (Pcap Analysis Library)","archived":false,"fork":false,"pushed_at":"2025-07-28T09:23:55.000Z","size":618,"stargazers_count":101,"open_issues_count":6,"forks_count":21,"subscribers_count":5,"default_branch":"master","last_synced_at":"2025-07-28T10:19:57.274Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/rusticata.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE-APACHE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2019-06-26T13:00:18.000Z","updated_at":"2025-07-28T09:24:01.000Z","dependencies_parsed_at":"2025-03-29T17:29:04.409Z","dependency_job_id":"7159296b-6f35-4fa9-9d43-339eee7f3b84","html_url":"https://github.com/rusticata/pcap-analyzer","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"purl":"pkg:github/rusticata/pcap-analyzer","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rusticata%2Fpcap-analyzer","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rusticata%2Fpcap-analyzer/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rusticata%2Fpcap-analyzer/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rusticata%2Fpcap-analyzer/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/rusticata","download_url":"https://codeload.github.com/rusticata/pcap-analyzer/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/rusticata%2Fpcap-analyzer/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267785735,"owners_count":24144120,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-29T02:00:12.549Z","response_time":2574,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2025-03-29T17:28:19.360Z","updated_at":"2025-07-30T00:33:24.323Z","avatar_url":"https://github.com/rusticata.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](./LICENSE-MIT)\n[![Apache License 2.0](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](./LICENSE-APACHE)\n[![Github CI](https://github.com/rusticata/pcap-analyzer/workflows/Continuous%20integration/badge.svg)](https://github.com/rusticata/pcap-analyzer/actions)\n\n# PAL (Pcap Analysis Library)\n\nPAL (Pcap Analysis Library) is a framework and a set of tools for Pcap file analysis, written in\nRust.\n\nThe main objectives are:\n\n- provide a robust and efficient framework for analyzing pcap files\n- provide tools to manipulate pcap files with a unifier abstraction\n- reconstruct network data progressively for network layers (Layer 2, 3, etc.) correctly, dealing\n  with common problems safely (fragmentation, missing data, encapsulation, etc.)\n- allow developing plugins easily, focusing only on the interesting content\n- allow plugins to interact with data at different network level (Layer 2, 3, application, etc.)\n- use Rust features like thread safety (to exploit parallelism), memory safety, zero-copy, etc.\n\n## Architecture\n\nPAL is split into several components:\n\n- `libpcap-tools`: a library providing support functions to manipulate pcap files\n- `libpcap-analyzer`: the main library, providing network data reconstruction, dispatch, and plugin\n  management. It also provides some plugins.\n- `pcap-analyzer`: the main executable to run plugins on pcap files\n- `pcap-rewrite`: a tool to rewrite a pcap file format and link type to another\n- `test-analyzer`: a similar tool to `pcap-analyzer`, with more debug plugins and verbosity (for ex. for debugging\n  plugins)\n- `explugin-example`: an example of plugin developed in a separate crate\n\n## Building pcap-analyzer\n\nUse `cargo` to build pcap-analyzer:\n\n```\n# release mode\ncargo build --release\n# debug mode\ncargo build\n```\n\n## Running pcap-analyzer\n\nJust run `pcap-analyzer` with the names of pcap files as arguments:\n\n```\npcap-analyzer file.pcap\npcap-analyzer -c config.toml file.pcap\n```\n\nThe `-p` option can be used to restrict the list of plugins to load.\n\nConcurrency level is set using the `-j` argument. Default is to 1 (no multithreading).\nThreading is useful when having many flows, so if the input file is small, or if it does not contain\nmany flows, it is best to leave it to 1.\nUse the value `0` to set the number of threads to the number of virtual CPUs.\n\nLogging is done using the `log` cargo crate, and will to the log file defined\nin configuration (`pcap-analyzer.log` by default).\nNote that in release mode, only messages with a severity of `warn` or more are displayed.\n\nTo get more debug info, use the `test-analyzer` tool. It provides the exact same features, but will\nbe more verbose, and will output logs to stderr. The `PCAP_ANALYZER_LOG` environment variable can be\nused to set the log level (and set concurrency to 1):\n\n```\nPCAP_ANALYZER_LOG=debug test-analyzer cargo run -p test-analyzer -- -j 1 -c conf/pcap-analyzer.conf file.pcap\n```\n\n## Plugins\n\nPlugins are modules that are selected during build, and can be activated during execution. They are\nembedded into the resulting library.\n\nNot all plugins are built by default, those that are not yet stable or have many dependencies are\nconditioned by a build feature. To build all plugins, activate the `all` feature, or select features\nindividually:\n\n```\ncargo build --all --all-features\n```\n\nYou can also edit `libpcap-analyzer/Cargo.toml` to edit the `default` feature.\n\nNote that due to limitations in the handling of features in workspaces by cargo, there seems to be\nno easy way to enable one feature only when building the package.\nAlso note that, due to the same limitations, `cargo run` will not use the features.\n\n*For the moment, the only \"stable\" method is to edit `libpcap-analyzer/Cargo.toml`.*\n\nPlugins can declare functions that will be called either when receiving data for a network layer, or\nfor some events:\n\n- layer 2: raw data (only if the pcap contains L2 data)\n- layer 3: raw data + ethernet type\n- layer 4: flow + l4 data + l4 payload (if l4 type is known/supported) + l3 data + ethertype + raw packet\n- creating of a flow\n- destruction of a flow\n\nFlows are created for every L4 communication. Flows use five-tuples (IP source and destination, L4\nprotocol, source and destination ports). If the protocol does not contain ports, they are set to 0.\n\nNote that functions can be called several times for a single packet. For example, in case of\nencapsulated data (like IP in IP), functions will be called in order (first, the outer data, then\nthe inner data).\n\n## Parallelism\n\nTo use parallelism, network packets have to be dispatched to worker threads. To ensure consistency,\nall packets from a single connexion have to be sent to the same worker, or this would create\nproblems like handling packets out-of-order.\n\nThe current implementation dispatches packets starting from the layer 3 (layer 2 is handled by the\nmain thread). The dispatch function is based on a symmetric hash function on IP parameters.\nAfter this dispatch, each worker thread handles its packets (in received order) and will reconstruct\nthe layer 2 (if present), calling plugins, then layer 3, call plugins, etc.\n\nTo ensure consistency, plugins are protected (using locks) before being called. The lock is done at\nthe plugin level, so only one handling function can be called at a time.\n\nIn particular:\n\n- if a plugin registers for several layers, the functions are guaranteed to be called in parsing\n  order (from outer data to inner data)\n- even if several packets are concurrently handled by several workers, a single plugin will not be\n  called concurrently. However, different plugins can execute concurrently.\n\n## Live mode\n\nExperimental support for live capture of packets is available, based on the `pcap` library.\n\nTo use it, install the required dependencies and enable the \"live\" feature during compilation:\n\n```\nsudo apt-get install libpcap-dev\ncargo build --features \"live\"\n```\n\nThe `--interface NAME` option (required) can be used after to specify the capture interface, and the\n`--filter FILTER_STRING` can be used to limit captured packets to those matching the filter:\n\n```\npcap-analyzer -c conf/pcap-analyzer.conf -i eth0\n```\n\nSet the configuration file for options relative to live capture.\n\n### Permissions\n\nLive mode requires `CAP_NET_RAW` (and possibly `CAP_NET_ADMIN` to list interfaces?), so the\nexecutable file requires either to be run using `sudo` or similar, or to have the file capabilities\nset.\n\nDeveloper note: to run the file with `sudo` when using `cargo`, use the `target.\"cfg(all())\".runner`\noption, for example:\n```\ncargo run --config 'target.\"cfg(all())\".runner=\"sudo -E\"' -- -c conf/pcap-analyzer.conf -i eth0\n```\n\n## Notes\n\n- pcap file parsing is completely reimplemented from scratch. This is the result of most existing\n  libraries lacking features, and the will to provide a unified abstraction to manipulate the\n  different subformats (pcap and pcapng, both in little and big-endian) and link types\n- pcap file read is done in a circular buffer (which size can be controlled using configuration).\n  Before each buffer refill, a synchronization is done to wait all workers to finish their current\n  jobs\n- the plugins are embedded into the main binary. Currently, there is no support for dynamic\n  libraries, due to the lack of support/stability by Rust\n\n## License\n\nLicensed under either of\n\n * Apache License, Version 2.0\n   ([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)\n * MIT license\n   ([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)\n\nat your option.\n\n## Contribution\n\nUnless you explicitly state otherwise, any contribution intentionally submitted\nfor inclusion in the work by you, as defined in the Apache-2.0 license, shall be\ndual licensed as above, without any additional terms or conditions.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frusticata%2Fpcap-analyzer","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Frusticata%2Fpcap-analyzer","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Frusticata%2Fpcap-analyzer/lists"}