{"id":44429561,"url":"https://github.com/cong-or/hud","last_synced_at":"2026-02-15T01:13:27.356Z","repository":{"id":333575506,"uuid":"1121096307","full_name":"cong-or/hud","owner":"cong-or","description":"Find code blocking your Tokio workers. eBPF-powered, no instrumentation.","archived":false,"fork":false,"pushed_at":"2026-02-12T13:11:59.000Z","size":6555,"stargazers_count":133,"open_issues_count":2,"forks_count":1,"subscribers_count":0,"default_branch":"main","last_synced_at":"2026-02-12T18:57:21.390Z","etag":null,"topics":["async","cli","debugging","ebpf","linux","observability","performance","profiler","rust","tokio","tui"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/cong-or.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null,"notice":null,"maintainers":null,"copyright":null,"agents":null,"dco":null,"cla":null}},"created_at":"2025-12-22T12:35:25.000Z","updated_at":"2026-02-12T13:08:45.000Z","dependencies_parsed_at":null,"dependency_job_id":null,"html_url":"https://github.com/cong-or/hud","commit_stats":null,"previous_names":["cong-or/hud"],"tags_count":13,"template":false,"template_full_name":null,"purl":"pkg:github/cong-or/hud","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cong-or%2Fhud","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cong-or%2Fhud/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cong-or%2Fhud/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cong-or%2Fhud/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/cong-or","download_url":"https://codeload.github.com/cong-or/hud/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/cong-or%2Fhud/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29463676,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-15T01:01:38.065Z","status":"ssl_error","status_checked_at":"2026-02-15T01:01:23.809Z","response_time":53,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["async","cli","debugging","ebpf","linux","observability","performance","profiler","rust","tokio","tui"],"created_at":"2026-02-12T12:14:06.748Z","updated_at":"2026-02-15T01:13:27.350Z","avatar_url":"https://github.com/cong-or.png","language":"Rust","funding_links":[],"categories":["Development tools"],"sub_categories":["Profiling"],"readme":"# hud\n\n[![CI](https://github.com/cong-or/hud/actions/workflows/ci.yml/badge.svg)](https://github.com/cong-or/hud/actions)\n[![License](https://img.shields.io/badge/license-MIT%2FApache--2.0-blue)](LICENSE)\n[![Linux 5.8+](https://img.shields.io/badge/Linux-5.8%2B-yellow?logo=linux)](docs/ARCHITECTURE.md)\n\nFind what's blocking your Tokio runtime. Zero-instrumentation eBPF profiler.\n\n\u003e **Linux only.** This tool uses eBPF, which is a Linux kernel feature. It does not work on macOS or Windows.\n\n![hud demo](docs/demo.gif)\n\n## The Problem\n\nTokio uses cooperative scheduling. Tasks yield at `.await` points, trusting that work between awaits is fast. When it isn't—CPU-heavy code, sync I/O, blocking locks—one task starves the rest.\n\nThese bugs are silent. No errors, no panics—just degraded throughput. hud makes them visible.\n\n## How It Works\n\nWatches the Linux scheduler via eBPF. When a worker thread experiences high OS-level scheduling latency (time the thread waits in the kernel run queue, not Tokio's task queue), captures a stack trace. High scheduling latency is a symptom of blocking—when one task monopolizes a worker, others queue up waiting.\n\n## Why hud?\n\nUnlike [tokio-console](https://github.com/tokio-rs/console) or [tokio-blocked](https://github.com/theduke/tokio-blocked), hud requires no code changes—attach to any running Tokio process.\n\n**Why not just use tokio-console?** It's the official tool and more accurate—it measures actual task poll durations. Use it if you can. But it requires adding `console-subscriber` and rebuilding.\n\n**What about Tokio's unstable blocking detection?** Compile with `RUSTFLAGS=\"--cfg tokio_unstable\"` and Tokio warns when task polls exceed a threshold. This catches the *blocker* directly, not victims—more accurate than hud. But it requires a rebuild, and only catches blocks exceeding the threshold during that run.\n\nhud exists for profiling without code changes or rebuilds—staging environments, load testing, quick triage of a running process, or confirming blocking is even the problem before investing in instrumentation.\n\n### When to use what\n\n| Tool | Best for | Trade-off |\n|------|----------|-----------|\n| **hud** | Quick triage of running processes | Measures symptoms, not direct cause |\n| **Tokio unstable detection** | Find the blocker directly | Requires rebuild with `tokio_unstable` |\n| **tokio-console** | Precise task poll times | Requires code instrumentation |\n| **perf + flamegraphs** | CPU profiling, broad analysis | Manual interpretation needed |\n| **Custom metrics** | Production monitoring | Must know where to instrument |\n\nUse hud to narrow down suspects, then dig deeper with instrumentation if needed.\n\n## Requirements\n\n**System:**\n- Linux 5.8+\n- x86_64 architecture\n- Root privileges\n\n**Your application needs debug symbols** (so hud can show function names):\n```toml\n# Cargo.toml\n[profile.release]\ndebug = true\nforce-frame-pointers = true\n```\n\n\u003e `debug = true` adds ~10-20% to binary size. `force-frame-pointers` adds ~1-2% runtime overhead. For production, you can swap in a debug-enabled binary temporarily for investigation.\n\n## Install\n\n**Option A: Pre-built binary** (no Rust toolchain needed)\n\n```bash\ncurl -L https://github.com/cong-or/hud/releases/latest/download/hud-linux-x86_64.tar.gz | tar xz\nsudo ./hud my-app\n```\n\n**Option B: Build from source**\n\n```bash\ngit clone https://github.com/cong-or/hud.git \u0026\u0026 cd hud\ncargo xtask build-ebpf --release \u0026\u0026 cargo build --release\nsudo ./target/release/hud my-app\n```\n\n## Usage\n\n```bash\n# Profile by process name\nsudo hud my-app\n\n# Profile by PID\nsudo hud --pid 1234\n\n# Custom blocking threshold (default: 5ms)\nsudo hud my-app --threshold 10   # less sensitive\nsudo hud my-app --threshold 1    # more sensitive\n\n# Rolling time window (only show last N seconds)\nsudo hud my-app --window 30      # metrics decay when load stops\n\n# Custom Tokio thread names are auto-detected. Override if needed:\nsudo hud my-app --workers my-io-worker\n\n# Headless mode (CI/scripting) - run for 60 seconds then exit\nsudo hud my-app --headless --export trace.json --duration 60\n```\n\nSee [Tuning](docs/TUNING.md) for threshold selection guide.\n\n## Demo\n\nTry hud with the included demo server (requires Option B):\n\n```bash\n# Build demo server - MUST be debug build (release inlines functions)\ncargo build --example demo-server\n./target/debug/examples/demo-server \u0026\n\n# Profile it (auto-detects PID and binary)\nsudo ./target/release/hud demo-server\n\n# Generate load (another terminal)\n./hud/examples/load.sh\n```\n\nThe demo server has intentionally blocking endpoints (`/hash`, `/compress`, `/read`, `/dns`). You'll see `bcrypt` and `blowfish` hotspots from the `/hash` endpoint, with `demo-server.rs` highlighted as the entry point in call traces.\n\n\u003e **Important**: The demo-server **must** be a debug build. Release builds aggressively inline functions, hiding your code from stack traces. If you don't see `demo-server.rs` in drilldowns, rebuild without `--release`.\n\nPress `Q` to quit hud.\n\n## Limitations\n\n- Measures scheduling latency (a *symptom* of blocking), not blocking directly\n- Captures the **victim's** stack, not the **blocker's**—if Task A blocks causing Task B to wait, you see Task B's stack. Look for patterns across multiple traces.\n- System CPU pressure can cause false positives—look for consistent, repeatable traces\n- Lock contention where threads sleep (not spin) may not appear\n- Tokio 1.x only. Worker detection tries the default thread name prefix, then stack-based classification (looks for Tokio scheduler frames), then largest thread group heuristic. Use `--workers \u003cprefix\u003e` to skip auto-detection\n- See [Troubleshooting](docs/TROUBLESHOOTING.md) for common issues\n\n## Docs\n\n- [Tuning](docs/TUNING.md) — Threshold selection, debugging workflow\n- [Exports](docs/EXPORTS.md) — JSON format, before/after analysis\n- [Troubleshooting](docs/TROUBLESHOOTING.md) — Common issues\n- [Architecture](docs/ARCHITECTURE.md) — How it works internally\n- [Development](docs/DEVELOPMENT.md) — Contributing\n\n## Further Reading\n\n- [Async: What is blocking?](https://ryhl.io/blog/async-what-is-blocking/) — Alice Ryhl's deep dive on blocking in async Rust\n- [tokio::task::spawn_blocking](https://docs.rs/tokio/latest/tokio/task/fn.spawn_blocking.html) — Offload blocking I/O to a thread pool\n- [rayon](https://docs.rs/rayon) — For parallelizable CPU work; call from within `spawn_blocking`, not directly from async code\n- [Reducing tail latencies with automatic cooperative task yielding](https://tokio.rs/blog/2020-04-preemption) — Tokio's approach to preemption\n\n## License\n\nMIT or Apache-2.0\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcong-or%2Fhud","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fcong-or%2Fhud","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fcong-or%2Fhud/lists"}