https://github.com/kirky-x/inklog

A high-performance, secure, and feature-rich logging solution built on Tokio. Features multi-target output (console, file, database, S3), AES-256-GCM encryption, Regex-based PII masking, log rotation & compression, structured logging, health monitoring, and metrics.
https://github.com/kirky-x/inklog
encryption loggin observability rust
Last synced: 5 months ago
JSON representation
Host: GitHub
URL: https://github.com/kirky-x/inklog
Owner: Kirky-X
License: mit
Created: 2026-01-17T18:26:33.000Z (5 months ago)
Default Branch: main
Last Pushed: 2026-01-18T06:13:03.000Z (5 months ago)
Last Synced: 2026-01-24T06:23:43.623Z (5 months ago)
Topics: encryption, loggin, observability, rust
Language: Rust
Homepage: https://crates.io/crates/inklog
Size: 580 KB
Stars: 0
Watchers: 0
Forks: 0
Open Issues: 0
Metadata Files:
- Readme: README.md
- Changelog: CHANGELOG.md
- Contributing: docs/CONTRIBUTING.md
- License: LICENSE
- Security: docs/SECURITY.md
Awesome Lists containing this project

README

          






  

  

    

  

  

  

    

  

  

  

    

  

  

  

    

  

  

  

    

  

  

  

    

  





  Enterprise-grade Rust Logging Infrastructure





  ✨ Features •

  🚀 Quick Start •

  📚 Documentation •

  💻 Examples •

  🤝 Contributing





---

### 🎯 A high-performance, secure, and feature-rich logging infrastructure built on Tokio

Inklog provides a **comprehensive** logging solution for enterprise applications:

| ⚡ High Performance | 🔒 Security First | 🌐 Multi-Target | 📊 Observability |

|:---------:|:----------:|:--------------:|:--------:|

| Async I/O with Tokio | AES-256-GCM encryption | Console, File, DB, S3 | Health monitoring |

| Batch writes & compression | Zeroized secret memory | Automatic rotation | Metrics & tracing |

```rust

use inklog::{InklogConfig, LoggerManager};

use std::path::PathBuf;

#[tokio::main]

async fn main() -> Result<(), Box> {

    let config = InklogConfig {

        file_sink: Some(inklog::FileSinkConfig {

            enabled: true,

            path: "logs/app.log".into(),

            max_size: "100MB".into(),

            compress: true,

            ..Default::default()

        }),

        ..Default::default()

    };

    let _logger = LoggerManager::with_config(config).await?;

    log::info!("Application started successfully");

    log::error!("Something went wrong with error details");

    Ok(())

}

```

---

## 📋 Table of Contents

📑 Table of Contents (Click to expand)

- [✨ Features](#features)

- [🚀 Quick Start](#quick-start)

  - [📦 Installation](#installation)

  - [💡 Basic Usage](#basic-usage)

  - [🔧 Advanced Configuration](#advanced-configuration)

- [🎨 Feature Flags](#feature-flags)

- [📚 Documentation](#documentation)

- [💻 Examples](#examples)

- [🏗️ Architecture](#architecture)

- [🔒 Security](#security)

- [🧪 Testing](#testing)

- [🤝 Contributing](#contributing)

- [📄 License](#license)

- [🙏 Acknowledgments](#acknowledgments)

---

## ✨ Features



| 🎯 Core Features | ⚡ Enterprise Features |

|:----------:|:----------:|

| Always Available | Optional |



### 🎯 Core Features (Always Available)

| Status | Feature | Description |

|:----:|------|------|

| ✅ | **Async I/O** | Tokio-powered non-blocking logging |

| ✅ | **Multi-Target Output** | Console, file, database, custom sinks |

| ✅ | **Structured Logging** | tracing ecosystem integration |

| ✅ | **Custom Formatting** | Template-based log format |

| ✅ | **File Rotation** | Size-based and time-based rotation |

| ✅ | **Data Masking** | Regex-based PII redaction |

| ✅ | **Health Monitoring** | Sink status and metrics tracking |

| ✅ | **CLI Tools** | decrypt, generate, validate commands |

### ⚡ Enterprise Features

| Status | Feature | Description |

|:----:|------|------|

| 🔍 | **Compression** | ZSTD, GZIP, Brotli, LZ4 support (`zstd`, `flate2`, etc.) |

| 🔒 | **Encryption** | AES-256-GCM file encryption (`aes-gcm`) |

| 🗄️ | **Database Sink** | PostgreSQL, MySQL, SQLite via Sea-ORM |

| ☁️ | **S3 Archive** | Cloud log archival with AWS SDK S3 (`aws` feature) |

| 📊 | **Parquet Export** | Analytics-ready log format (`parquet` feature) |

| 🌐 | **HTTP Endpoint** | Axum-based health check server (`http` feature) |

| 📅 | **Scheduled Tasks** | Cron-based archive scheduling |

| 🔧 | **CLI Tools** | Utility commands for log management (`cli` feature) |

| 📝 | **TOML Config** | External configuration support (`confers` feature) |

### 📦 Feature Presets

| Preset | Features | Use Case |

|------|------|----------|

| minimal | No optional features | Core logging only |

| standard | `http`, `cli` | Standard development setup |

| full | All default features | Production-ready logging |

---

## 🚀 Quick Start

### 📦 Installation

Add this to your `Cargo.toml`:

```toml

[dependencies]

inklog = "0.1"

```

For full feature set:

```toml

[dependencies]

inklog = { version = "0.1", features = ["default"] }

```

### 💡 Basic Usage



#### 🎬 5-Minute Quick Start



**Step 1: Initialize Logger**

```rust

use inklog::LoggerManager;

#[tokio::main]

async fn main() -> Result<(), Box> {

    let _logger = LoggerManager::new().await?;

    log::info!("Logger initialized");

    Ok(())

}

```

**Step 2: Log Messages**

```rust

use inklog::LoggerManager;

#[tokio::main]

async fn main() -> Result<(), Box> {

    let _logger = LoggerManager::new().await?;

    log::trace!("Trace message");

    log::debug!("Debug message");

    log::info!("Info message");

    log::warn!("Warning message");

    log::error!("Error message");

    Ok(())

}

```

**Step 3: File Logging**

```rust

use inklog::{FileSinkConfig, InklogConfig, LoggerManager};

let config = InklogConfig {

    file_sink: Some(FileSinkConfig {

        enabled: true,

        path: "logs/app.log".into(),

        max_size: "10MB".into(),

        rotation_time: "daily".into(),

        keep_files: 7,

        compress: true,

        ..Default::default()

    }),

    ..Default::default()

};

let _logger = LoggerManager::with_config(config).await?;

```

**Step 4: Database Logging**

```rust

use inklog::{DatabaseSinkConfig, InklogConfig, config::DatabaseDriver};

let config = InklogConfig {

    database_sink: Some(DatabaseSinkConfig {

        enabled: true,

        driver: DatabaseDriver::SQLite,

        url: "sqlite://logs/app.db".to_string(),

        pool_size: 5,

        batch_size: 100,

        flush_interval_ms: 1000,

        ..Default::default()

    }),

    ..Default::default()

};

let _logger = LoggerManager::with_config(config).await?;

```

### 🔧 Advanced Configuration

#### Encrypted File Logging

```rust

use inklog::{FileSinkConfig, InklogConfig};

// Set encryption key from environment

std::env::set_var("INKLOG_ENCRYPTION_KEY", "base64-encoded-32-byte-key");

let config = InklogConfig {

    file_sink: Some(FileSinkConfig {

        enabled: true,

        path: "logs/encrypted.log.enc".into(),

        max_size: "10MB".into(),

        encrypt: true,

        encryption_key_env: Some("INKLOG_ENCRYPTION_KEY".into()),

        compress: false, // Don't compress encrypted logs

        ..Default::default()

    }),

    ..Default::default()

};

let _logger = LoggerManager::with_config(config).await?;

```

#### S3 Cloud Archiving

```rust

use inklog::{InklogConfig, S3ArchiveConfig};

let config = InklogConfig {

    s3_archive: Some(S3ArchiveConfig {

        enabled: true,

        bucket: "my-log-bucket".to_string(),

        region: "us-west-2".to_string(),

        archive_interval_days: 7,

        local_retention_days: 30,

        prefix: "logs/".to_string(),

        compression: inklog::archive::CompressionType::Zstd,

        ..Default::default()

    }),

    ..Default::default()

};

let manager = LoggerManager::with_config(config).await?;

manager.start_archive_service().await?;

```

#### Custom Log Format

```rust

use inklog::{InklogConfig, config::GlobalConfig};

let format_string = "[{timestamp}] [{level:>5}] {target} - {message} | {file}:{line}";

let config = InklogConfig {

    global: GlobalConfig {

        level: "debug".into(),

        format: format_string.to_string(),

        masking_enabled: true,

        ..Default::default()

    },

    ..Default::default()

};

let _logger = LoggerManager::with_config(config).await?;

```

---

## 🎨 Feature Flags

### Default Features

```toml

inklog = "0.1"  # Includes: aws, http, cli

```

### Optional Features

```toml

# Cloud & Storage

inklog = { version = "0.1", features = [

    "aws",        # AWS S3 archive support

] }

# HTTP Server

inklog = { version = "0.1", features = [

    "http",       # Axum HTTP health endpoint

] }

# CLI Tools

inklog = { version = "0.1", features = [

    "cli",        # decrypt, generate, validate commands

] }

# Configuration

inklog = { version = "0.1", features = [

    "confers",    # TOML configuration support

] }

# Development

inklog = { version = "0.1", features = [

    "test-local", # Local testing mode

    "debug",     # Additional security audit logging

] }

```

### Feature Details

| Feature | Dependencies | Description |

|---------|-------------|-------------|

| **aws** | aws-sdk-s3, aws-config, aws-types | AWS S3 cloud archive |

| **http** | axum | HTTP health check endpoint |

| **cli** | clap, glob, toml | Command-line utilities |

| **confers** | confers, toml | External TOML configuration support |

| **test-local** | - | Local testing mode |

| **debug** | - | Security audit logging |

---

## 📚 Documentation







📘 API Reference






Complete API docs





💻 Examples






Working code examples





📖 Guides






In-depth guides



### 📖 Additional Resources

| Resource | Description |

|----------|-------------|

| 📘 [API Reference](https://docs.rs/inklog) | Complete API documentation on docs.rs |

| 🏗️ [Architecture](docs/ARCHITECTURE.md) | System architecture and design decisions |

| 🔒 [Security](docs/SECURITY.md) | Security best practices and features |

| 📦 [Examples](examples/) | Working code examples for all features |

---

## 💻 Examples



### 💡 Real-world Examples



#### 📝 Basic Logging

```rust

use inklog::LoggerManager;

#[tokio::main]

async fn main() -> Result<(), Box> {

    let _logger = LoggerManager::new().await?;

    log::info!("Application started");

    log::error!("An error occurred: {}", err);

    Ok(())

}

```

#### 📁 File Logging with Rotation

```rust

use inklog::{FileSinkConfig, InklogConfig, LoggerManager};

let config = InklogConfig {

    file_sink: Some(FileSinkConfig {

        enabled: true,

        path: "logs/app.log".into(),

        max_size: "10MB".into(),

        rotation_time: "daily".into(),

        keep_files: 7,

        compress: true,

        ..Default::default()

    }),

    ..Default::default()

};

let _logger = LoggerManager::with_config(config).await?;

```

#### 🔒 Encrypted Logging

```rust

use inklog::{FileSinkConfig, InklogConfig};

std::env::set_var("INKLOG_ENCRYPTION_KEY", "base64-encoded-key");

let config = InklogConfig {

    file_sink: Some(FileSinkConfig {

        enabled: true,

        path: "logs/encrypted.log".into(),

        encrypt: true,

        encryption_key_env: Some("INKLOG_ENCRYPTION_KEY".into()),

        ..Default::default()

    }),

    ..Default::default()

};

let _logger = LoggerManager::with_config(config).await?;

```

#### 🗄️ Database Logging

```rust

use inklog::{DatabaseSinkConfig, InklogConfig, config::DatabaseDriver};

let config = InklogConfig {

    database_sink: Some(DatabaseSinkConfig {

        enabled: true,

        driver: DatabaseDriver::PostgreSQL,

        url: "postgresql://localhost/logs".to_string(),

        pool_size: 10,

        batch_size: 100,

        flush_interval_ms: 1000,

        ..Default::default()

    }),

    ..Default::default()

};

let _logger = LoggerManager::with_config(config).await?;

```

#### ☁️ S3 Cloud Archive

```rust

use inklog::{InklogConfig, S3ArchiveConfig};

let config = InklogConfig {

    s3_archive: Some(S3ArchiveConfig {

        enabled: true,

        bucket: "my-log-bucket".to_string(),

        region: "us-west-2".to_string(),

        archive_interval_days: 7,

        local_retention_days: 30,

        prefix: "logs/".to_string(),

        compression: inklog::archive::CompressionType::Zstd,

        ..Default::default()

    }),

    ..Default::default()

};

let manager = LoggerManager::with_config(config).await?;

manager.start_archive_service().await?;

```

#### 🏥 HTTP Health Endpoint

```rust

use axum::{routing::get, Json, Router};

use inklog::LoggerManager;

use std::sync::Arc;

let logger = Arc::new(LoggerManager::new().await?);

let app = Router::new().route(

    "/health",

    get({

        let logger = logger.clone();

        || async move { Json(logger.get_health_status()) }

    }),

);

// Start HTTP server...

```

#### 🎨 Custom Format

```rust

use inklog::{InklogConfig, config::GlobalConfig};

let format_string = "[{timestamp}] [{level:>5}] {target} - {message}";

let config = InklogConfig {

    global: GlobalConfig {

        level: "debug".into(),

        format: format_string.to_string(),

        masking_enabled: true,

        ..Default::default()

    },

    ..Default::default()

};

let _logger = LoggerManager::with_config(config).await?;

```

#### 🔍 Data Masking

```rust

use inklog::{InklogConfig, config::GlobalConfig};

let config = InklogConfig {

    global: GlobalConfig {

        level: "info".into(),

        format: "{timestamp} {level} {message}".to_string(),

        masking_enabled: true,  // Enable PII masking

        ..Default::default()

    },

    ..Default::default()

};

let _logger = LoggerManager::with_config(config).await?;

// Sensitive data will be automatically masked

log::info!("User email: user@example.com");

// Output: User email: ***@***.***

```



**[📂 View all examples →](examples/)**



---

## 🏗️ Architecture



### 🏗️ System Architecture



```

┌─────────────────────────────────────────────────┐

│           Application Layer                    │

│  (Your code using log! macros)             │

└────────────────┬────────────────────────────┘

                 │

┌────────────────▼────────────────────────────┐

│         Inklog API Layer                  │

│  - LoggerManager, LoggerBuilder          │

│  - Configuration management               │

│  - Health monitoring                     │

└────────────────┬────────────────────────────┘

                 │

┌────────────────▼────────────────────────────┐

│         Sink Abstraction Layer             │

│  - ConsoleSink                          │

│  - FileSink (rotation, compression)     │

│  - DatabaseSink (batch writes)           │

│  - AsyncFileSink                        │

│  - RingBufferedFileSink                 │

└────────────────┬────────────────────────────┘

                 │

┌────────────────▼────────────────────────────┐

│         Core Processing Layer              │

│  - Log formatting & templates            │

│  - Data masking (PII redaction)         │

│  - Encryption (AES-256-GCM)             │

│  - Compression (ZSTD, GZIP, Brotli)    │

└────────────────┬────────────────────────────┘

                 │

┌────────────────▼────────────────────────────┐

│         Concurrency & I/O                 │

│  - Tokio async runtime                  │

│  - Crossbeam channels                  │

│  - Rayon parallel processing            │

└────────────────┬────────────────────────────┘

                 │

┌────────────────▼────────────────────────────┐

│         Storage & External Services        │

│  - Filesystem                          │

│  - Database (PostgreSQL, MySQL, SQLite)  │

│  - AWS S3 (cloud archive)              │

│  - Parquet (analytics)                 │

└───────────────────────────────────────────┘

```

### Layer-by-Layer Explanation

**Application Layer**

- Application code uses standard `log!` macros from the `log` crate

- Compatible with existing Rust logging patterns

**Inklog API Layer**

- `LoggerManager`: Main orchestrator for all logging operations

- `LoggerBuilder`: Fluent builder pattern for configuration

- Health status tracking and metrics collection

**Sink Abstraction Layer**

- Multiple sink implementations for different output targets

- Console output for development

- File output with rotation, compression, and encryption

- Database output with batch writes (PostgreSQL, MySQL, SQLite)

- Async and buffered file sinks for high-throughput scenarios

**Core Processing Layer**

- Template-based log formatting

- Regex-based PII data masking (emails, SSNs, credit cards)

- AES-256-GCM encryption for sensitive logs

- Multiple compression algorithms (ZSTD, GZIP, Brotli, LZ4)

**Concurrency & I/O Layer**

- Tokio async runtime for non-blocking I/O

- Crossbeam channels for inter-task communication

- Rayon for CPU-intensive parallel processing

**Storage & External Services Layer**

- Local filesystem access

- Database connectivity via Sea-ORM

- AWS S3 integration for cloud archival

- Parquet format for analytics workflows

---

## 🔒 Security



### 🛡️ Security Features



Inklog is built with security as a top priority:

#### 🔒 Encryption

- **AES-256-GCM**: Military-grade encryption for log files

- **Key Management**: Environment variable-based key injection

- **Zeroized Memory**: Secrets are securely cleared after use via `zeroize` crate

- **SHA-256 Hashing**: Integrity verification for encrypted logs

#### 🎭 Data Masking

- **Regex-Based Patterns**: Automatic PII detection and redaction

- **Email Masking**: `user@example.com` → `***@***.***`

- **SSN Masking**: Credit card and social security number redaction

- **Custom Patterns**: Configurable regex patterns for sensitive data

#### 🔐 Secure Key Handling

```rust

// Set encryption key securely from environment

std::env::set_var("INKLOG_ENCRYPTION_KEY", "base64-encoded-32-byte-key");

// Key is automatically zeroized after use

// Never hardcode keys in your application

```

#### 🛡️ Security Best Practices

- **No hardcoded secrets**: Keys loaded from environment variables

- **Minimal privileged operations**: Only necessary file/database access

- **Audit logging**: Debug feature for security audit trails

- **Compliance-ready**: Supports GDPR, HIPAA, PCI-DSS logging requirements

---

## 🧪 Testing



### 🎯 Run Tests



```bash

# Run all tests with default features

cargo test --all-features

# Run tests with specific features

cargo test --features "aws,http,cli"

# Run tests in release mode

cargo test --release

# Run benchmarks

cargo bench

```

### Test Coverage

Inklog targets **95%+ code coverage**:

```bash

# Generate coverage report

cargo tarpaulin --out Html --all-features

```

### Linting and Formatting

```bash

# Format code

cargo fmt --all

# Check formatting without changes

cargo fmt --all -- --check

# Run Clippy (warnings as errors)

cargo clippy --all-targets --all-features -- -D warnings

```

### Security Audit

```bash

# Run cargo deny for security checks

cargo deny check

# Check for advisories

cargo deny check advisories

# Check for banned licenses

cargo deny check bans

```

### Integration Tests

```bash

# Run integration tests

cargo test --test '*'

# Run with Docker services (PostgreSQL, MySQL)

docker-compose up -d

cargo test --all-features

docker-compose down

```

---

## 🤝 Contributing



Contributions are welcome! Please see [CONTRIBUTING.md](CONTRIBUTING.md) for guidelines.



### Development Setup

```bash

# Clone repository

git clone https://github.com/Kirky-X/inklog.git

cd inklog

# Install pre-commit hooks (if available)

./scripts/install-pre-commit.sh

# Run tests

cargo test --all-features

# Run linter

cargo clippy --all-features

# Format code

cargo fmt --all

```

### Pull Request Process

1. Fork the repository

2. Create a feature branch (`git checkout -b feature/amazing-feature`)

3. Make your changes

4. Run tests and ensure all pass (`cargo test --all-features`)

5. Run clippy and fix warnings (`cargo clippy --all-features`)

6. Commit your changes (`git commit -m 'Add amazing feature'`)

7. Push to the branch (`git push origin feature/amazing-feature`)

8. Open a Pull Request

### Code Style

- Follow Rust naming conventions (snake_case for variables, PascalCase for types)

- Use `thiserror` for error types

- Use `anyhow` for error contexts

- Add doc comments to all public APIs

- Run `cargo fmt` before committing

---

## 📄 License



This project is dual-licensed under **MIT / Apache-2.0**:

[![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE-MIT)

[![License: Apache-2.0](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](LICENSE-APACHE)



---

## 🙏 Acknowledgments



### 🌟 Built on Excellent Tools



Inklog wouldn't be possible without these amazing projects:

- [tracing](https://github.com/tokio-rs/tracing) - The foundation of Rust structured logging

- [tokio](https://tokio.rs/) - Async runtime for Rust

- [Sea-ORM](https://www.sea-ql.org/SeaORM/) - Async ORM for database operations

- [AWS SDK for Rust](https://github.com/awslabs/aws-sdk-rust) - AWS S3 integration

- [axum](https://github.com/tokio-rs/axum) - Web framework for HTTP endpoints

- [serde](https://serde.rs/) - Serialization framework

- The entire Rust ecosystem for amazing tools and libraries

---

## 📞 Support







📋 Issues






Report bugs and issues





💬 Discussions






Ask questions and share ideas





🐙 GitHub






View source code



---

## ⭐ Star History



[![Star History Chart](https://api.star-history.com/svg?repos=Kirky-X/inklog&type=Date)](https://star-history.com/#Kirky-X/inklog&Date)



---



### 💝 Support This Project

If you find this project useful, please consider giving it a ⭐️!

**Built with ❤️ by Inklog Team**

---

**[⬆ Back to Top](#inklog)**

---

_{© 2026 Inklog Project. All rights reserved.}
ecosyste.ms

Data

Tools

Indexes

Applications

Experiments

Awesome

https://github.com/kirky-x/inklog

Awesome Lists containing this project

README
