https://github.com/mablr/trace2e
TracE2E: A distributed traceability framework that provides provenance recording and compliance enforcement for processes' I/O operations. Track data flows across your distributed infrastructure with seamless middleware integration, enabling explainability, and policy enforcement for sensitive data handling.
https://github.com/mablr/trace2e
decentralized io middleware rust traceability
Last synced: 7 months ago
JSON representation
TracE2E: A distributed traceability framework that provides provenance recording and compliance enforcement for processes' I/O operations. Track data flows across your distributed infrastructure with seamless middleware integration, enabling explainability, and policy enforcement for sensitive data handling.
- Host: GitHub
- URL: https://github.com/mablr/trace2e
- Owner: mablr
- License: mit
- Created: 2024-04-11T12:19:40.000Z (almost 2 years ago)
- Default Branch: master
- Last Pushed: 2025-06-03T09:55:29.000Z (8 months ago)
- Last Synced: 2025-06-03T20:53:01.604Z (8 months ago)
- Topics: decentralized, io, middleware, rust, traceability
- Language: Rust
- Homepage:
- Size: 389 KB
- Stars: 2
- Watchers: 2
- Forks: 0
- Open Issues: 0
-
Metadata Files:
- Readme: README.md
- License: LICENSE
Awesome Lists containing this project
README
# TracE2E - Distributed Traceability Middleware
A distributed traceability framework that provides provenance recording and compliance enforcement for processes' I/O operations in distributed systems.
## Overview
TracE2E framework is composed of several key components:
- **trace2e_middleware**: The core middleware service that handles provenance recording and compliance enforcement
- **trace2e_client**: Client library to interact with the middleware
- **stde2e**: Standard library wrapper providing TracE2E integration using `trace2e_client`
- **tokioe2e**: We propose a patch for `tokio` library providing TracE2E integration using `trace2e_client` like for `stde2e`
## Key Features
TracE2E provides a comprehensive suite of capabilities to enable end-to-end provenance recording and compliance enforcement for I/O operations in distributed systems:
- Process Observation: Track and Mediate file and network I/O operations seamlessly at the process level.
- Distributed Provenance Tracking: Provide explainability and accountability across distributed systems with a unified data flow model.
- Compliance Enforcement: policies can be applied to resources taking into account the provenance, the type, and the location of the resource.
## Architecture
The system works through three main components:
1. **Middleware Service**: Runs on each node where traceability is enforced, providing:
- Process-to-Middleware (P2M) communication
- Middleware-to-Middleware (M2M) communication
- Traceability enforcement
2. **Client Library**: Wraps standard I/O operations to:
- Process-to-Middleware (P2M) communication
- Register files and network streams opened by the process
- Request authorization for I/O operations
- Report operation results
3. **I/O Library Wrapper and Patches**: to provide TracE2E integration using `trace2e_client`:
- `stde2e`: Standard library wrapper
- `tokioe2e`: Patch for `tokio` library
## Getting Started
### Prerequisites
- Rust
- Protocol Buffers compiler
### Building from Source
1. Clone the repository:
```bash
git clone https://github.com/mablr/trace2e
cd trace2e
```
2. Build the middleware in release mode to enable optimizations:
```bash
cargo build -r
./target/release/trace2e_middleware
```
The middleware will listen on `[::]:8080` by default.
### Benchmarking
To benchmark the middleware internal traceability server, run the following command:
```bash
cargo bench -p trace2e_middleware
```
To benchmark the `trace2e_client` and the `stde2e` wrapper, run the following commands:
```bash
./target/release/trace2e_middleware &
cargo bench -p trace2e_client
cargo bench -p stde2e
```
## Project Structure
- **/proto**: Protocol buffer definitions for communication between components
- **/trace2e_middleware**: Core middleware implementation
- **/trace2e_client**: Middleware client communication library implementation
- **/stde2e**: Standard library wrapper providing TracE2E integration using `trace2e_client`
- **/patches**: TracE2E integration patches for various frameworks
## License
This project is licensed under the terms found in the LICENSE file in the root directory.