{"id":27914980,"url":"https://github.com/conradludgate/tokio-supervisor","last_synced_at":"2025-05-06T15:33:15.700Z","repository":{"id":268034847,"uuid":"851154785","full_name":"conradludgate/tokio-supervisor","owner":"conradludgate","description":"Reports when tokio runtime threads are blocking","archived":false,"fork":false,"pushed_at":"2024-09-04T09:54:30.000Z","size":27,"stargazers_count":64,"open_issues_count":0,"forks_count":1,"subscribers_count":2,"default_branch":"main","last_synced_at":"2025-05-04T23:52:42.106Z","etag":null,"topics":["async","rust","tokio"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":null,"status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/conradludgate.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":null,"code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2024-09-02T14:24:58.000Z","updated_at":"2025-04-11T12:56:19.000Z","dependencies_parsed_at":"2024-12-14T00:15:33.632Z","dependency_job_id":"3350933d-5b42-408d-9db4-a5a14d97fc44","html_url":"https://github.com/conradludgate/tokio-supervisor","commit_stats":null,"previous_names":["conradludgate/tokio-supervisor"],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/conradludgate%2Ftokio-supervisor","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/conradludgate%2Ftokio-supervisor/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/conradludgate%2Ftokio-supervisor/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/conradludgate%2Ftokio-supervisor/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/conradludgate","download_url":"https://codeload.github.com/conradludgate/tokio-supervisor/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":252713045,"owners_count":21792421,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["async","rust","tokio"],"created_at":"2025-05-06T15:33:10.151Z","updated_at":"2025-05-06T15:33:15.688Z","avatar_url":"https://github.com/conradludgate.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# tokio-supervisor\n\nWork in progress\n\n## What is this?\n\nThis is a library that hooks into your tokio runtime (using unstable features) to monitor your runtime\nthreads for suspicious tasks that are blocking the thread. It currently only works on Linux and MacOS.\n\nIn the future, it will also attempt to spawn some new worker threads to pick up the slack to try\nand improve efficiency and unblock your application in the rare event that all your tasks end up blocking.\n\n## How does it work?\n\nUsing `RuntimeMetrics::worker_poll_count`, we can detect when a worker is no longer making progress. If the poll\ncount does not increase after some duration, it is flagged as blocking (it could also be parked, which we also test for). If the thread\nis detected as blocking, we send a signal to that thread.\n\nAdditionally, we set up a global channel and signal handler on all threads to respond to this signal.\nIt handles the signal by capturing a backtrace and sending it over the channel. By capturing the backtrace,\nit makes it much easier to debug what is causing the runtime worker to become blocked.\n\n## How will it handle spawning new workers?\n\nTokio has a mechanism built in to spawn a new temporary worker thread called `block_in_place`. Unfortunately\nit will reclaim the worker state once the block_in_place method completes. We would need to move this logic\nso that tokio only reclaims the worker state once the task unblocks itself.\n\n## Example\n\n```rust\n#[tokio::main(flavor = \"multi_thread\", worker_threads = 2)]\nasync fn main() {\n    // Create a new supervisor and attach it to the current runtime.\n    // Spawn the supervisor thread to sample all workers every 100ms.\n    Supervisor::new(\u0026tokio::runtime::Handle::current()).spawn(Duration::from_millis(100));\n\n    // Run some workload in the runtime.\n    let svc = axum::Router::new()\n        .route(\"/fast\", get(|| async {}))\n        // will block the runtime when this API is hit.\n        .route(\n            \"/slow\",\n            get(|| async { std::thread::sleep(Duration::from_secs(5)) }),\n        )\n        .with_state(())\n        .into_make_service();\n    axum::serve(TcpListener::bind(\"0.0.0.0:8123\").await.unwrap(), svc)\n        .await\n        .unwrap()\n}\n```\n\nCalling `http://localhost:8123/fast` shows nothing. Calling `http://localhost:8123/slow` prints\n\n```\nworker thread 1 is blocked (673097 == 673097)\nbacktrace::backtrace::libunwind::trace::h99d4c3ef5d4daf63\n -\u003e \"/Users/conrad/.cargo/registry/src/index.crates.io-6f17d22bba15001f/backtrace-0.3.73/src/backtrace/libunwind.rs\"\nbacktrace::backtrace::trace_unsynchronized::h9b3887e61be83d18\n -\u003e \"/Users/conrad/.cargo/registry/src/index.crates.io-6f17d22bba15001f/backtrace-0.3.73/src/backtrace/mod.rs\"\ntokio_supervisor::tests::get_backtrace2::hedb14cb7a09bf091\n -\u003e \"/Users/conrad/Documents/code/tokio-supervisor/src/lib.rs\"\nsignal_hook_registry::handler::hffda5a27bfe9fd9b\n -\u003e \"/Users/conrad/.cargo/registry/src/index.crates.io-6f17d22bba15001f/signal-hook-registry-1.4.2/src/lib.rs\"\n__simple_esappend\nstd::sys::pal::unix::thread::Thread::sleep::ha9be65893542756d\n -\u003e \"/rustc/3f5fd8dd41153bc5fdca9427e9e05be2c767ba23/library/std/src/sys/pal/unix/thread.rs\"\nstd::thread::sleep::h68e6720733935c56\n -\u003e \"/rustc/3f5fd8dd41153bc5fdca9427e9e05be2c767ba23/library/std/src/thread/mod.rs\"\ntokio_supervisor::tests::it_works::{{closure}}::{{closure}}::{{closure}}::h7baafb1d89b51864\n -\u003e \"/Users/conrad/Documents/code/tokio-supervisor/src/lib.rs\"\n\u003cF as axum::handler::Handler\u003c((),),S\u003e\u003e::call::{{closure}}::hdab8fd20326fa8e7\n -\u003e \"/Users/conrad/.cargo/registry/src/index.crates.io-6f17d22bba15001f/axum-0.7.5/src/handler/mod.rs\"\n\u003ccore::pin::Pin\u003cP\u003e as core::future::future::Future\u003e::poll::h236b098ffeea1b99\n -\u003e \"/rustc/3f5fd8dd41153bc5fdca9427e9e05be2c767ba23/library/core/src/future/future.rs\"\n\u003cfutures_util::future::future::map::Map\u003cFut,F\u003e as core::future::future::Future\u003e::poll::hfdd9947fda4e1606\n -\u003e \"/Users/conrad/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.30/src/future/future/map.rs\"\n\u003cfutures_util::future::future::Map\u003cFut,F\u003e as core::future::future::Future\u003e::poll::h23cd2e63b5ba6313\n -\u003e \"/Users/conrad/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.30/src/lib.rs\"\n\u003caxum::handler::future::IntoServiceFuture\u003cF\u003e as core::future::future::Future\u003e::poll::hf95ed4d49ff0ba63\n -\u003e \"/Users/conrad/.cargo/registry/src/index.crates.io-6f17d22bba15001f/axum-0.7.5/src/macros.rs\"\n\u003cF as futures_core::future::TryFuture\u003e::try_poll::h1ad9def16d43a26e\n -\u003e \"/Users/conrad/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-core-0.3.30/src/future.rs\"\n\u003cfutures_util::future::try_future::into_future::IntoFuture\u003cFut\u003e as core::future::future::Future\u003e::poll::h76645dc11ea53258\n -\u003e \"/Users/conrad/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.30/src/future/try_future/into_future.rs\"\n\u003cfutures_util::future::future::map::Map\u003cFut,F\u003e as core::future::future::Future\u003e::poll::hf796f107af9e2d56\n -\u003e \"/Users/conrad/.cargo/registry/src/index.crates.io-6f17d22bba15001f/futures-util-0.3.30/src/future/future/map.rs\n```\n\nThere's a lot of noise here, but if we filter down the output, we do see our `std::thread::sleep` inside an axum handler\n\n```\nworker thread 1 is blocked (673097 == 673097)\n...\nstd::thread::sleep::h68e6720733935c56\n -\u003e \"/rustc/3f5fd8dd41153bc5fdca9427e9e05be2c767ba23/library/std/src/thread/mod.rs\"\ntokio_supervisor::tests::it_works::{{closure}}::{{closure}}::{{closure}}::h7baafb1d89b51864\n -\u003e \"/Users/conrad/Documents/code/tokio-supervisor/src/lib.rs\"\n\u003cF as axum::handler::Handler\u003c((),),S\u003e\u003e::call::{{closure}}::hdab8fd20326fa8e7\n -\u003e \"/Users/conrad/.cargo/registry/src/index.crates.io-6f17d22bba15001f/axum-0.7.5/src/handler/mod.rs\"\n...\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fconradludgate%2Ftokio-supervisor","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fconradludgate%2Ftokio-supervisor","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fconradludgate%2Ftokio-supervisor/lists"}