{"id":24298262,"url":"https://github.com/krypt0nn/miyabi_scheduler","last_synced_at":"2026-02-07T11:31:57.818Z","repository":{"id":271073039,"uuid":"912327307","full_name":"krypt0nn/miyabi_scheduler","owner":"krypt0nn","description":"Rust thread pool scheduler with extra specific abilities","archived":false,"fork":false,"pushed_at":"2025-01-06T14:21:51.000Z","size":364,"stargazers_count":1,"open_issues_count":0,"forks_count":0,"subscribers_count":1,"default_branch":"main","last_synced_at":"2025-10-10T17:55:30.319Z","etag":null,"topics":["rust","rust-library","scheduler","thread-pool"],"latest_commit_sha":null,"homepage":"","language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/krypt0nn.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2025-01-05T09:00:58.000Z","updated_at":"2025-01-06T14:21:51.000Z","dependencies_parsed_at":null,"dependency_job_id":"b8824098-1992-457c-b43e-28bf39ee2ebf","html_url":"https://github.com/krypt0nn/miyabi_scheduler","commit_stats":null,"previous_names":["krypt0nn/miyabi_scheduler"],"tags_count":6,"template":false,"template_full_name":null,"purl":"pkg:github/krypt0nn/miyabi_scheduler","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/krypt0nn%2Fmiyabi_scheduler","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/krypt0nn%2Fmiyabi_scheduler/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/krypt0nn%2Fmiyabi_scheduler/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/krypt0nn%2Fmiyabi_scheduler/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/krypt0nn","download_url":"https://codeload.github.com/krypt0nn/miyabi_scheduler/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/krypt0nn%2Fmiyabi_scheduler/sbom","scorecard":null,"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29193585,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-07T07:37:03.739Z","status":"ssl_error","status_checked_at":"2026-02-07T07:37:03.029Z","response_time":63,"last_error":"SSL_connect returned=1 errno=0 peeraddr=140.82.121.6:443 state=error: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["rust","rust-library","scheduler","thread-pool"],"created_at":"2025-01-16T20:54:38.872Z","updated_at":"2026-02-07T11:31:57.795Z","avatar_url":"https://github.com/krypt0nn.png","language":"Rust","funding_links":[],"categories":[],"sub_categories":[],"readme":"# Miyabi Scheduler\n\n`miyabi_scheduler` is a specific threads scheduling library for rust where\nyou expect to have two main types of tasks: \"thin\" and \"thick\"\n(internally called \"normal\" and \"exclusive\" / \"locking\"). This scheduler\nallows you to gain \"exclusive lock\" for one specific task and all the tasks\nwhich will be spawned by it.\n\nNormally tasks queue looks like this:\n\n\u003cimg src=\"./images/normal_tasks_queue.png\" /\u003e\n\nScheduler uses 2 workers to process them one by another, with all 3 sitting\nin the buffer. However, sometimes you want to spawn new tasks while processing\nthe current ones! And sometimes you want these new tasks to use data owned\nby its ancestor task and drop it later to not to waste user's RAM. That's what\nI call an \"exclusive lock\" or \"exclusive task\": you let scheduler know that\nyou really want to process this one specific task using all the available\npowers, and only then switch to the other tasks. That's how it looks like:\n\n\u003cimg src=\"./images/tasks_queue_with_exclusive_lock.png\" /\u003e\n\nScheduler will spawn a new thread for this one exclusive task to process it there,\nand use all available workers to process all the new tasks which will be spawned\nby the exclusive one. New thread is needed to ensure that exclusive tasks\nwill not be bounded by the amount of available workers. Otherwise, say\nyour scheduler has only 1 worker and you have 2 exclusive tasks: you would\nhave a little problem processing such a queue. Thus new thread is a necessary evil.\n\nIt's important to note that the scheduler will not be unlocked until\nthe exclusive task's *context* (special struct to spawn new tasks)\nis dropped. So if you move it in another thread and forget to drop\nat some point - you must count on your own! Make a copy of the scheduler's\noriginal context struct if you want to spawn new tasks in it.\n\nSometimes your exclusive tasks can be nested. This is also handled properly\nby this library, by making a recursive lock (so you can't repeat the proces\ninfinitely, but I really didn't want to implement some loop logic for this).\nThat's how it would look like:\n\n\u003cimg src=\"./images/tasks_queue_with_nested_exclusive_lock.png\" /\u003e\n\nTask 1 will spawn 2, 3, and 6. Then, while processing 2, we will lock\nscheduler *again* and process 4 and 5. Then, once 3's context is dropped,\nunlock second lock and continue processing 6. Then unlock 2's lock and\ncontinue processing 7 and 8.\n\n## Example: substring search in files within a directory\n\nImagine we want to find a substring in any files within a directory.\nOr naturally do any computation task that can easily be paralleled.\nYou can use any threadpool library for this, implement your own one,\nor use async runtime for this goal. And of course you can use\nMiyabi Scheduler! It's not what it was made for, but as a general\nscheduler of course it can solve this problem.\n\nYou can run the following example with this command:\n\n```bash\ncargo run --example find_substring -- \u003cpath to a folder\u003e 'your substring'\n```\n\nCode (from the `examples/find_substring.rs` file):\n\n```rust\nuse std::io::Read;\nuse std::path::PathBuf;\nuse std::fs::File;\n\nuse miyabi_scheduler::prelude::*;\n\n// It's important to silence all the errors or process them\n// in a different place to not to crash threads where tasks\n// are executed.\n\n/// Return scheduler task to scan given folder for a substring.\nfn task_scan_dir(path: PathBuf, substring: String) -\u003e Task {\n    Box::new(move |context| {\n        if let Ok(entries) = path.read_dir() {\n            for entry in entries.flatten() {\n                let path = entry.path();\n\n                if path.is_dir() {\n                    // Schedule exclusive task for directory scanning.\n                    // This will force Miyabi Scheduler to execute this function\n                    // (directory scanning) in a new thread and use all the workers\n                    // to process the tasks it produces (either scan other folders\n                    // or scan found files).\n                    //\n                    // From technical perspective it's absolutely pointless.\n                    // But it's an example project, so I use it just to demonstrate\n                    // that this thing exists.\n                    let _ = context.schedule_exclusive(task_scan_dir(path, substring.clone()));\n                }\n\n                else if path.is_file() {\n                    // Schedule normal task for file reading. Not different\n                    // from any other schedulers.\n                    let _ = context.schedule(task_scan_file(path, substring.clone()));\n                }\n            }\n        }\n    })\n}\n\n/// Return scheduler task to scan given file for a substring.\nfn task_scan_file(path: PathBuf, substring: String) -\u003e Task {\n    Box::new(move |context| {\n        let mut current = 0;\n\n        let total = path.metadata()\n            .expect(\"Failed to read file's metadata\")\n            .len();\n\n        let mut file = File::open(\u0026path)\n            .expect(\"Failed to open file\");\n\n        context.named_scope(\"scan_file\", move |scope| {\n            let mut buf = [0; 4096];\n\n            let substring = substring.as_bytes();\n            let substring_len = substring.len();\n\n            loop {\n                let Ok(n) = file.read(\u0026mut buf) else {\n                    break;\n                };\n\n                if n == 0 {\n                    break;\n                }\n\n                // Sure there's a chance we have half substring in this buffer,\n                // and another half in the next buffer. But I don't want to\n                // overcomplicate the simple example.\n                if n \u003e substring.len() {\n                    for i in 0..n - substring_len {\n                        if \u0026buf[i..i + substring_len] == substring {\n                            let _ = scope.status(format!(\"Found substring in file {path:#?}, offset {}\", current + i as u64));\n                        }\n                    }\n                }\n\n                current += n as u64;\n\n                let _ = scope.progress(current, total);\n            }\n        });\n    })\n}\n\nfn main() {\n    let path = std::env::args().nth(1)\n        .map(PathBuf::from)\n        .expect(\"Directory path expected, got nothing\");\n\n    let substring = std::env::args().nth(2)\n        .expect(\"Search substring expected, got nothing\");\n\n    if !path.is_dir() {\n        panic!(\"Directory path expected: {path:#?}\");\n    }\n\n    let mut scheduler = Scheduler::new(8, 1024);\n\n    scheduler.schedule_exclusive(task_scan_dir(path, substring))\n        .expect(\"Failed to schedule directory scanning\");\n\n    let mut last_fraction = 0.0;\n\n    while scheduler.is_alive() {\n        scheduler.update(|report, handler| {\n            if let ScopeReport::Status { status, .. } = report {\n                println!(\"{status}\");\n            }\n\n            if let Some(status) = handler.named(\"scan_file\") {\n                // Print progress with 1% step.\n                if status.progress.is_some() \u0026\u0026 status.fraction \u003e last_fraction \u0026\u0026 status.fraction - last_fraction \u003e= 0.01 {\n                    last_fraction = status.fraction;\n\n                    println!(\n                        \"Scanning files: {}%, elapsed {:.2} seconds, remaining {:.2} seconds\",\n                        last_fraction * 100.0,\n                        status.elapsed_time.as_millis() as f64 / 1000.0,\n                        status.estimate_time.as_millis() as f64 / 1000.0\n                    );\n                }\n            }\n        }).expect(\"Failed to update scheduler\");\n    }\n}\n```\n\n*Named after [Hoshimi Miyabi](https://zenless-zone-zero.fandom.com/wiki/Hoshimi_Miyabi) from Zenless Zone Zero 🙏*\n\n\u003cimg src=\"./images/miyabi.png\"\u003e\n\n*Image was taken from [here](https://www.pinterest.com/pin/350858627237946126)*\n\nAuthor: [Nikita Podvirnyi](https://github.com/krypt0nn)\\\nLicensed under [MIT](LICENSE)\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkrypt0nn%2Fmiyabi_scheduler","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fkrypt0nn%2Fmiyabi_scheduler","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fkrypt0nn%2Fmiyabi_scheduler/lists"}