https://github.com/rusticata/pcap-analyzer
PAL (Pcap Analysis Library)
https://github.com/rusticata/pcap-analyzer
Last synced: about 2 months ago
JSON representation
PAL (Pcap Analysis Library)
- Host: GitHub
- URL: https://github.com/rusticata/pcap-analyzer
- Owner: rusticata
- License: apache-2.0
- Created: 2019-06-26T13:00:18.000Z (almost 6 years ago)
- Default Branch: master
- Last Pushed: 2024-11-26T15:59:04.000Z (6 months ago)
- Last Synced: 2025-03-29T17:34:06.532Z (about 2 months ago)
- Language: Rust
- Size: 570 KB
- Stars: 98
- Watchers: 5
- Forks: 20
- Open Issues: 5
-
Metadata Files:
- Readme: README.md
- License: LICENSE-APACHE
Awesome Lists containing this project
README
[](./LICENSE-MIT)
[](./LICENSE-APACHE)
[](https://github.com/rusticata/pcap-analyzer/actions)
# PAL (Pcap Analysis Library)
PAL (Pcap Analysis Library) is a framework and a set of tools for Pcap file analysis, written in
Rust.
The main objectives are:
- provide a robust and efficient framework for analyzing pcap files
- provide tools to manipulate pcap files with a unifier abstraction
- reconstruct network data progressively for network layers (Layer 2, 3, etc.) correctly, dealing
with common problems safely (fragmentation, missing data, encapsulation, etc.)
- allow developing plugins easily, focusing only on the interesting content
- allow plugins to interact with data at different network level (Layer 2, 3, application, etc.)
- use Rust features like thread safety (to exploit parallelism), memory safety, zero-copy, etc.
## Architecture
PAL is split into several components:
- `libpcap-tools`: a library providing support functions to manipulate pcap files
- `libpcap-analyzer`: the main library, providing network data reconstruction, dispatch, and plugin
management. It also provides some plugins.
- `pcap-analyzer`: the main executable to run plugins on pcap files
- `pcap-rewrite`: a tool to rewrite a pcap file format and link type to another
- `test-analyzer`: a similar tool to `pcap-analyzer`, with more debug plugins and verbosity (for ex. for debugging
plugins)
- `explugin-example`: an example of plugin developed in a separate crate
## Building pcap-analyzer
Use `cargo` to build pcap-analyzer:
```
# release mode
cargo build --release
# debug mode
cargo build
```
## Running pcap-analyzer
Just run `pcap-analyzer` with the names of pcap files as arguments:
```
pcap-analyzer file.pcap
pcap-analyzer -c config.toml file.pcap
```
The `-p` option can be used to restrict the list of plugins to load.
Concurrency level is set using the `-j` argument. Default is to 1 (no multithreading).
Threading is useful when having many flows, so if the input file is small, or if it does not contain
many flows, it is best to leave it to 1.
Use the value `0` to set the number of threads to the number of virtual CPUs.
Logging is done using the `log` cargo crate, and will to the log file defined
in configuration (`pcap-analyzer.log` by default).
Note that in release mode, only messages with a severity of `warn` or more are displayed.
To get more debug info, use the `test-analyzer` tool. It provides the exact same features, but will
be more verbose, and will output logs to stderr. The `PCAP_ANALYZER_LOG` environment variable can be
used to set the log level (and set concurrency to 1):
```
PCAP_ANALYZER_LOG=debug test-analyzer cargo run -p test-analyzer -- -j 1 -c conf/pcap-analyzer.conf file.pcap
```
## Plugins
Plugins are modules that are selected during build, and can be activated during execution. They are
embedded into the resulting library.
Not all plugins are built by default, those that are not yet stable or have many dependencies are
conditioned by a build feature. To build all plugins, activate the `all` feature, or select features
individually:
```
cargo build --all --all-features
```
You can also edit `libpcap-analyzer/Cargo.toml` to edit the `default` feature.
Note that due to limitations in the handling of features in workspaces by cargo, there seems to be
no easy way to enable one feature only when building the package.
Also note that, due to the same limitations, `cargo run` will not use the features.
*For the moment, the only "stable" method is to edit `libpcap-analyzer/Cargo.toml`.*
Plugins can declare functions that will be called either when receiving data for a network layer, or
for some events:
- layer 2: raw data (only if the pcap contains L2 data)
- layer 3: raw data + ethernet type
- layer 4: flow + l4 data + l4 payload (if l4 type is known/supported) + l3 data + ethertype + raw packet
- creating of a flow
- destruction of a flow
Flows are created for every L4 communication. Flows use five-tuples (IP source and destination, L4
protocol, source and destination ports). If the protocol does not contain ports, they are set to 0.
Note that functions can be called several times for a single packet. For example, in case of
encapsulated data (like IP in IP), functions will be called in order (first, the outer data, then
the inner data).
## Parallelism
To use parallelism, network packets have to be dispatched to worker threads. To ensure consistency,
all packets from a single connexion have to be sent to the same worker, or this would create
problems like handling packets out-of-order.
The current implementation dispatches packets starting from the layer 3 (layer 2 is handled by the
main thread). The dispatch function is based on a symmetric hash function on IP parameters.
After this dispatch, each worker thread handles its packets (in received order) and will reconstruct
the layer 2 (if present), calling plugins, then layer 3, call plugins, etc.
To ensure consistency, plugins are protected (using locks) before being called. The lock is done at
the plugin level, so only one handling function can be called at a time.
In particular:
- if a plugin registers for several layers, the functions are guaranteed to be called in parsing
order (from outer data to inner data)
- even if several packets are concurrently handled by several workers, a single plugin will not be
called concurrently. However, different plugins can execute concurrently.
## Notes
- pcap file parsing is completely reimplemented from scratch. This is the result of most existing
libraries lacking features, and the will to provide a unified abstraction to manipulate the
different subformats (pcap and pcapng, both in little and big-endian) and link types
- pcap file read is done in a circular buffer (which size can be controlled using configuration).
Before each buffer refill, a synchronization is done to wait all workers to finish their current
jobs
- the plugins are embedded into the main binary. Currently, there is no support for dynamic
libraries, due to the lack of support/stability by Rust
## License
Licensed under either of
* Apache License, Version 2.0
([LICENSE-APACHE](LICENSE-APACHE) or http://www.apache.org/licenses/LICENSE-2.0)
* MIT license
([LICENSE-MIT](LICENSE-MIT) or http://opensource.org/licenses/MIT)
at your option.
## Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted
for inclusion in the work by you, as defined in the Apache-2.0 license, shall be
dual licensed as above, without any additional terms or conditions.