{"id":35214877,"url":"https://github.com/usetero/policy-rs","last_synced_at":"2026-02-18T04:03:07.467Z","repository":{"id":330512412,"uuid":"1122409399","full_name":"usetero/policy-rs","owner":"usetero","description":"Rust implementation for the Policy spec","archived":false,"fork":false,"pushed_at":"2026-02-11T20:30:57.000Z","size":323,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":0,"default_branch":"master","last_synced_at":"2026-02-12T04:39:35.008Z","etag":null,"topics":["observability","policy","rust","telemetry"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/usetero.png","metadata":{"files":{"readme":"README.md","changelog":"CHANGELOG.md","contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-24T17:01:11.000Z","updated_at":"2026-02-11T20:30:46.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/usetero/policy-rs","commit_stats":null,"previous_names":["usetero/policy-rs"],"tags_count":13,"template":false,"template_full_name":null,"purl":"pkg:github/usetero/policy-rs","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/usetero%2Fpolicy-rs","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/usetero%2Fpolicy-rs/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/usetero%2Fpolicy-rs/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/usetero%2Fpolicy-rs/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/usetero","download_url":"https://codeload.github.com/usetero/policy-rs/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/usetero%2Fpolicy-rs/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29567616,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-18T00:47:08.760Z","status":"online","status_checked_at":"2026-02-18T02:00:09.468Z","response_time":162,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["observability","policy","rust","telemetry"],"created_at":"2025-12-29T21:26:36.140Z","updated_at":"2026-02-18T04:03:07.425Z","avatar_url":"https://github.com/usetero.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# policy-rs\n\nRust implementation of the\n[Tero Policy Specification](https://github.com/usetero/policy) for\nhigh-performance log policy evaluation and transformation.\n\nAnother implementation of this specification is available in\n[Tero Edge](https://github.com/usetero/edge), a Zig-based observability edge\nruntime, providing the policy evaluation engine for filtering, sampling, and\ntransforming telemetry data.\n\n## Features\n\n- **High-performance pattern matching** using\n  [Hyperscan](https://github.com/intel/hyperscan) for parallel regex evaluation\n- **Policy-based log filtering** with keep, drop, sample, and rate-limit actions\n- **Log transformations** including field removal, redaction, renaming, and\n  addition\n- **Multiple policy providers** with live reload support\n- **Zero-allocation field access** through the `Matchable` trait\n- **Async-first design** built on Tokio\n\n## Installation\n\nAdd to your `Cargo.toml`:\n\n```toml\n[dependencies]\npolicy-rs = { git = \"https://github.com/usetero/policy-rs\" }\n```\n\n## Quick Start\n\n```rust\nuse policy_rs::{EvaluateResult, FileProvider, PolicyEngine, PolicyRegistry, Matchable, LogFieldSelector};\n\n#[tokio::main]\nasync fn main() -\u003e Result\u003c(), Box\u003cdyn std::error::Error\u003e\u003e {\n    // Create registry and load policies\n    let registry = PolicyRegistry::new();\n    let provider = FileProvider::new(\"policies.json\");\n    registry.subscribe(\u0026provider)?;\n\n    // Create engine and get snapshot\n    let engine = PolicyEngine::new();\n    let snapshot = registry.snapshot();\n\n    // Evaluate a log record\n    let log = MyLogRecord::new(\"Error: connection timeout\", \"ERROR\");\n    let result = engine.evaluate(\u0026snapshot, \u0026log).await?;\n\n    match result {\n        EvaluateResult::NoMatch =\u003e println!(\"Pass through\"),\n        EvaluateResult::Keep { policy_id, .. } =\u003e println!(\"Keep: {}\", policy_id),\n        EvaluateResult::Drop { policy_id } =\u003e println!(\"Drop: {}\", policy_id),\n        EvaluateResult::Sample { keep, .. } =\u003e println!(\"Sampled: {}\", keep),\n        EvaluateResult::RateLimit { allowed, .. } =\u003e println!(\"Rate limited: {}\", allowed),\n    }\n\n    Ok(())\n}\n```\n\n## Core Concepts\n\n### Policy Registry\n\nThe `PolicyRegistry` manages policies from multiple providers and maintains an\nimmutable snapshot for lock-free evaluation:\n\n```rust\nlet registry = PolicyRegistry::new();\n\n// Subscribe to a file-based provider (auto-reloads on changes)\nlet provider = FileProvider::new(\"policies.json\");\nregistry.subscribe(\u0026provider)?;\n\n// Or register a custom provider\nlet handle = registry.register_provider();\nhandle.update(vec![policy1, policy2]);\n\n// Get immutable snapshot for evaluation\nlet snapshot = registry.snapshot();\n```\n\n### Policy Engine\n\nThe `PolicyEngine` evaluates logs against compiled policies using Hyperscan for\npattern matching:\n\n```rust\nlet engine = PolicyEngine::new();\nlet snapshot = registry.snapshot();\n\n// Read-only evaluation\nlet result = engine.evaluate(\u0026snapshot, \u0026log).await?;\n\n// Evaluation with transformations applied\nlet result = engine.evaluate_and_transform(\u0026snapshot, \u0026mut log).await?;\n```\n\n### Evaluation Results\n\n```rust\npub enum EvaluateResult {\n    /// No policies matched - pass through unchanged\n    NoMatch,\n    /// Matched policy says keep\n    Keep { policy_id: String, transformed: bool },\n    /// Matched policy says drop\n    Drop { policy_id: String },\n    /// Matched policy says sample (percentage-based)\n    Sample { policy_id: String, percentage: f64, keep: bool, transformed: bool },\n    /// Matched policy says rate limit (count-based)\n    RateLimit { policy_id: String, allowed: bool, transformed: bool },\n}\n```\n\n## Implementing the Traits\n\nTo evaluate your log types, implement the `Matchable` trait. For transformation\nsupport, also implement `Transformable`.\n\n### Matchable Trait\n\nThe `Matchable` trait provides zero-allocation field access for pattern\nmatching:\n\n```rust\nuse policy_rs::{Matchable, LogFieldSelector};\nuse policy_rs::proto::tero::policy::v1::LogField;\n\nstruct MyLogRecord {\n    body: String,\n    severity: String,\n    attributes: HashMap\u003cString, String\u003e,\n}\n\nimpl Matchable for MyLogRecord {\n    fn get_field(\u0026self, field: \u0026LogFieldSelector) -\u003e Option\u003c\u0026str\u003e {\n        match field {\n            LogFieldSelector::Simple(log_field) =\u003e match log_field {\n                LogField::Body =\u003e Some(\u0026self.body),\n                LogField::SeverityText =\u003e Some(\u0026self.severity),\n                _ =\u003e None,\n            },\n            LogFieldSelector::LogAttribute(key) =\u003e {\n                self.attributes.get(key).map(|s| s.as_str())\n            },\n            LogFieldSelector::ResourceAttribute(key) =\u003e None,\n            LogFieldSelector::ScopeAttribute(key) =\u003e None,\n        }\n    }\n}\n```\n\n### Transformable Trait\n\nThe `Transformable` trait enables field mutations when using\n`evaluate_and_transform`:\n\n```rust\nuse policy_rs::{Transformable, LogFieldSelector};\n\nimpl Transformable for MyLogRecord {\n    fn remove_field(\u0026mut self, field: \u0026LogFieldSelector) -\u003e bool {\n        match field {\n            LogFieldSelector::LogAttribute(key) =\u003e {\n                self.attributes.remove(key).is_some()\n            },\n            _ =\u003e false,\n        }\n    }\n\n    fn redact_field(\u0026mut self, field: \u0026LogFieldSelector, replacement: \u0026str) -\u003e bool {\n        match field {\n            LogFieldSelector::LogAttribute(key) =\u003e {\n                if self.attributes.contains_key(key) {\n                    self.attributes.insert(key.clone(), replacement.to_string());\n                    true\n                } else {\n                    false\n                }\n            },\n            _ =\u003e false,\n        }\n    }\n\n    fn rename_field(\u0026mut self, from: \u0026LogFieldSelector, to: \u0026str, upsert: bool) -\u003e bool {\n        if let LogFieldSelector::LogAttribute(key) = from {\n            if let Some(value) = self.attributes.remove(key) {\n                if upsert || !self.attributes.contains_key(to) {\n                    self.attributes.insert(to.to_string(), value);\n                    return true;\n                }\n            }\n        }\n        false\n    }\n\n    fn add_field(\u0026mut self, field: \u0026LogFieldSelector, value: \u0026str, upsert: bool) -\u003e bool {\n        match field {\n            LogFieldSelector::LogAttribute(key) =\u003e {\n                if upsert || !self.attributes.contains_key(key) {\n                    self.attributes.insert(key.clone(), value.to_string());\n                    true\n                } else {\n                    false\n                }\n            },\n            _ =\u003e false,\n        }\n    }\n}\n```\n\n## Advanced Usage\n\n### Custom Policy Providers\n\nImplement `PolicyProvider` to load policies from custom sources:\n\n```rust\nuse policy_rs::{PolicyProvider, PolicyCallback, Policy, PolicyError};\n\nstruct MyProvider {\n    // Your state here\n}\n\nimpl PolicyProvider for MyProvider {\n    fn load(\u0026self, callback: \u0026PolicyCallback) -\u003e Result\u003c(), PolicyError\u003e {\n        let policies = self.fetch_policies()?;\n        callback.update(policies);\n        Ok(())\n    }\n}\n\n// Use with the registry\nlet registry = PolicyRegistry::new();\nlet provider = MyProvider::new();\nregistry.subscribe(\u0026provider)?;\n```\n\n### Policy Statistics\n\nTrack policy hit/miss rates and transform statistics:\n\n```rust\nlet snapshot = registry.snapshot();\n\nfor entry in snapshot.iter() {\n    let stats = entry.stats.snapshot();\n\n    println!(\"Policy: {}\", entry.policy.id());\n    println!(\"  Matches: {} hits, {} misses\", stats.match_hits, stats.match_misses);\n    println!(\"  Remove: {} hits, {} misses\", stats.remove.0, stats.remove.1);\n    println!(\"  Redact: {} hits, {} misses\", stats.redact.0, stats.redact.1);\n    println!(\"  Rename: {} hits, {} misses\", stats.rename.0, stats.rename.1);\n    println!(\"  Add: {} hits, {} misses\", stats.add.0, stats.add.1);\n}\n```\n\n### Multiple Providers\n\nCombine policies from multiple sources:\n\n```rust\nlet registry = PolicyRegistry::new();\n\n// File-based policies\nlet file_provider = FileProvider::new(\"local-policies.json\");\nregistry.subscribe(\u0026file_provider)?;\n\n// Programmatic policies\nlet handle = registry.register_provider();\nhandle.update(vec![\n    create_emergency_drop_policy(),\n    create_rate_limit_policy(),\n]);\n\n// All policies are merged in the snapshot\nlet snapshot = registry.snapshot();\n```\n\n### Configuration-Based Providers\n\nUse the config module to define providers in JSON/TOML configuration files. The\n`ProviderConfig` type is designed to be embedded in your application's config:\n\n```rust\nuse policy_rs::config::{ProviderConfig, register_providers};\nuse policy_rs::PolicyRegistry;\nuse serde::Deserialize;\n\n#[derive(Deserialize)]\nstruct AppConfig {\n    service_name: String,\n    policy_providers: Vec\u003cProviderConfig\u003e,\n}\n\n// Parse your app config\nlet config: AppConfig = serde_json::from_str(r#\"{\n    \"service_name\": \"my-app\",\n    \"policy_providers\": [\n        {\n            \"id\": \"local\",\n            \"type\": \"file\",\n            \"path\": \"policies.json\"\n        },\n        {\n            \"id\": \"remote\",\n            \"type\": \"http\",\n            \"url\": \"https://api.example.com/policies\",\n            \"headers\": [\n                { \"name\": \"Authorization\", \"value\": \"Bearer token123\" }\n            ],\n            \"poll_interval_secs\": 60\n        }\n    ]\n}\"#)?;\n\n// Register all providers at once\nlet registry = PolicyRegistry::new();\nregister_providers(\u0026config.policy_providers, \u0026registry)?;\n```\n\n#### Provider Config Format\n\nEach provider configuration has a `type` field that determines the provider:\n\n**File Provider:**\n\n```json\n{\n  \"id\": \"local-policies\",\n  \"type\": \"file\",\n  \"path\": \"policies.json\"\n}\n```\n\n**HTTP Provider** (requires `http` feature):\n\n```json\n{\n  \"id\": \"remote-policies\",\n  \"type\": \"http\",\n  \"url\": \"https://api.example.com/policies\",\n  \"headers\": [{ \"name\": \"Authorization\", \"value\": \"Bearer token\" }],\n  \"poll_interval_secs\": 60,\n  \"content_type\": \"application/json\"\n}\n```\n\n**gRPC Provider** (requires `grpc` feature):\n\n```json\n{\n  \"id\": \"grpc-policies\",\n  \"type\": \"grpc\",\n  \"endpoint\": \"https://grpc.example.com:443\"\n}\n```\n\nYou can also parse just the provider list directly:\n\n```rust\nlet providers: Vec\u003cProviderConfig\u003e = serde_json::from_str(r#\"[\n    { \"id\": \"file\", \"type\": \"file\", \"path\": \"policies.json\" }\n]\"#)?;\n```\n\n### Transform Order\n\nWhen using `evaluate_and_transform`, transformations are applied in a fixed\norder:\n\n1. **Remove** - Delete fields\n2. **Redact** - Replace field values with placeholders\n3. **Rename** - Rename fields to new keys\n4. **Add** - Add new fields\n\nTransforms from all matching policies are applied, not just the winning policy.\n\n## Policy Format\n\nPolicies are defined using the\n[Tero Policy protobuf schema](https://github.com/usetero/policy). Example JSON:\n\n```json\n{\n  \"id\": \"drop-debug-logs\",\n  \"name\": \"Drop Debug Logs\",\n  \"enabled\": true,\n  \"target\": {\n    \"log\": {\n      \"match\": [\n        {\n          \"logField\": \"SEVERITY_TEXT\",\n          \"regex\": \"DEBUG|TRACE\"\n        }\n      ],\n      \"keep\": \"none\"\n    }\n  }\n}\n```\n\n### Keep Values\n\n- `\"all\"` - Keep all matching logs\n- `\"none\"` - Drop all matching logs\n- `\"50%\"` - Sample 50% of matching logs\n- `\"100/s\"` - Rate limit to 100 logs per second\n- `\"1000/m\"` - Rate limit to 1000 logs per minute\n\n### Match Fields\n\n- `logField` - Simple fields: `BODY`, `SEVERITY_TEXT`, `TRACE_ID`, `SPAN_ID`,\n  etc.\n- `logAttribute` - Log attributes by key\n- `resourceAttribute` - Resource attributes by key\n- `scopeAttribute` - Scope attributes by key\n\n### Match Types\n\n- `exact` - Exact string match\n- `regex` - Regular expression match\n- `exists` - Field existence check\n\n## Examples\n\nSee the `examples/` directory:\n\n- `basic_usage.rs` - Load policies and evaluate logs\n- `transforms.rs` - Apply log transformations\n- `multiple_providers.rs` - Combine multiple policy sources\n- `custom_provider.rs` - Implement a custom provider\n- `config_providers.rs` - Configure providers via JSON config\n\nRun examples with:\n\n```sh\ncargo run --example basic_usage\ncargo run --example transforms\ncargo run --example config_providers\n```\n\n## License\n\nApache-2.0\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fusetero%2Fpolicy-rs","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fusetero%2Fpolicy-rs","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fusetero%2Fpolicy-rs/lists"}