{"id":13502893,"url":"https://github.com/duo-rs/duo","last_synced_at":"2025-03-29T12:33:15.181Z","repository":{"id":38410055,"uuid":"443997479","full_name":"duo-rs/duo","owner":"duo-rs","description":"A lightweight Logging and Tracing observability solution for Rust, built with Apache Arrow, Apache Parquet and Apache DataFusion.","archived":false,"fork":false,"pushed_at":"2024-09-18T07:02:50.000Z","size":2632,"stargazers_count":67,"open_issues_count":1,"forks_count":7,"subscribers_count":4,"default_branch":"master","last_synced_at":"2024-09-19T06:40:20.232Z","etag":null,"topics":["apache-arrow","apache-parquet","datafusion","logging","observability","rust","tracing"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/duo-rs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":".github/FUNDING.yml","license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null},"funding":{"github":["Folyd"]}},"created_at":"2022-01-03T08:58:41.000Z","updated_at":"2024-09-18T07:02:53.000Z","dependencies_parsed_at":"2022-08-25T02:50:16.087Z","dependency_job_id":"1dd463bd-1e42-4457-87bc-0fba1a49a383","html_url":"https://github.com/duo-rs/duo","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/duo-rs%2Fduo","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/duo-rs%2Fduo/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/duo-rs%2Fduo/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/duo-rs%2Fduo/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/duo-rs","download_url":"https://codeload.github.com/duo-rs/duo/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":222495264,"owners_count":16993289,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["apache-arrow","apache-parquet","datafusion","logging","observability","rust","tracing"],"created_at":"2024-07-31T22:02:28.709Z","updated_at":"2024-10-31T22:31:00.653Z","avatar_url":"https://github.com/duo-rs.png","language":"Rust","funding_links":["https://github.com/sponsors/Folyd"],"categories":["Rust"],"sub_categories":[],"readme":"# Duo\n\n[![Crates.io](https://img.shields.io/crates/v/duo.svg)](https://crates.io/crates/duo)\n![Crates.io](https://img.shields.io/crates/d/duo)\n[![license-mit](https://img.shields.io/badge/license-MIT-yellow.svg)](./LICENSE)\n[![dependency status](https://deps.rs/repo/github/duo-rs/duo/status.svg)](https://deps.rs/repo/github/duo-rs/duo)\n\n**A lightweight Logging and Tracing observability solution for Rust, built with [Apache Arrow](https://arrow.apache.org), [Apache Parquet](https://parquet.apache.org) and [Apache DataFusion](https://datafusion.apache.org).**\n\n\u003e **Notice: this project is in the experimental stage and not production-ready. Use at your own risk.**\n\n## What is duo?\n\nDuo is an easy-to-use observability solution that provides both logging and tracing capabilities for Rust applications. While traditional observability solutions are powerful (such as [ELK](https://elastic.co), [jaegertracing](https://jaegertracing.io), etc), it is also complex to deploy and maintain. Duo aimed to provide a less-powerful but complete set of observability features, with extremely simple deployment and maintenance.\n\nThis project was inspired by [tracing](https://github.com/tokio-rs/tracing) and [console](https://github.com/tokio-rs/console), which mainly consist of multiple components:\n\n- **duo-api** - a wire protocol for logging and tracing data. The wire format is defined using gRPC and protocol buffers.\n- **duo-subscriber** - instrumentation for collecting logging and tracing data from a process and exposing it over the wire format. `duo-subscriber` crate in this repository contains an implementation of the instrumentation-side API as a [tracing-subscriber](https://crates.io/crates/tracing-subscriber) [Layer](https://docs.rs/tracing-subscriber/latest/tracing_subscriber/layer/trait.Layer.html), for projects using Tokio and tracing.\n- **duo-ui** - the web UI for duo.\n- **duo-server** - the aggregating server to collect tracing and logging data and interact with duo web UI.\n\n## Get started\n\n### Installation\n\n```\ncargo install duo\n```\n\nRun `duo`.\n\n```\n$ duo\n\ngRPC server listening on http://127.0.0.1:6000\n\nWeb server listening on http://127.0.0.1:3000\n```\n\nOpen https://127.0.0.1:3000 at your local browser to wait application report data.\n\n### Application\n\n```toml\nduo-subscriber = \"0.1\"\n```\n\n```rs\n#[tokio::main]\nasync fn main() {\n    let fmt_layer = fmt::layer();\n    let uri = Uri::from_static(\"http://127.0.0.1:6000\");\n    let (duo_layer, handle) = DuoLayer::with_handle(\"example\", uri).await;\n    tracing_subscriber::registry()\n        .with(fmt_layer)\n        .with(duo_layer)\n        .init();\n\n    tracing::debug!(\"Bootstrap...\");\n    foo();\n\n    handle.await.unwrap();\n}\n```\n\n\u003e For more example, please see [examples directory](./duo-subscriber/examples/).\n\nRun your application then check the http://127.0.0.1:3000 to see the tracing data.\n\n### Logging UI\n\n![](./duo-ui-logging.png)\n\n### Tracing UI\n\nCurrently trace view is based on [Jaeger UI](https://www.jaegertracing.io), we'll rewrite it with Svelte in the future.\n\n![](./duo-ui-tracing.png)\n\n## Roadmap\n\n- [x] Support tracing diagnosing with Jaeger UI.\n\n- [x] Build duo web UI.\n\n- [x] Support logging diagnosing.\n\n- [x] Support arrow-ipc WAL.\n\n- [x] Batch sync WAL to parquet files.\n\n- [x] Support Object Store.\n\n- [ ] Integrate Apache Iceberg?\n\n- [ ] Support OpenTelemetry specification, aimed to be a lightweight OpenTelemetry backend.\n\n## Why called duo?\n\nDuo is mainly a musical terminology meaning a musical composition for two performers in which the performers have equal importance to the piece, often a composition involving two singers or two pianists.\n\nThe famous duo band is [Brooklyn Duo](https://www.youtube.com/c/BrooklynDuo), you can visit this video ([Canon in D (Pachelbel's Canon) - Cello \u0026 Piano](https://www.youtube.com/watch?v=Ptk_1Dc2iPY)) to learn more about them.\n\n![](https://i.ytimg.com/vi/Ptk_1Dc2iPY/maxresdefault.jpg)\n\nI personally think the logging and tracing have equal importance to observability, they are just like a duo band to help you diagnose your application.\n\n## License\n\nThis project is licensed under the [MIT license](./LICENSE).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fduo-rs%2Fduo","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fduo-rs%2Fduo","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fduo-rs%2Fduo/lists"}