{"id":18964243,"url":"https://github.com/janeliascicomp/aitch","last_synced_at":"2026-02-27T18:11:38.771Z","repository":{"id":161282027,"uuid":"610903112","full_name":"JaneliaSciComp/aitch","owner":"JaneliaSciComp","description":"local scheduler for heterogenous jobs","archived":false,"fork":false,"pushed_at":"2024-02-20T18:35:30.000Z","size":44,"stargazers_count":3,"open_issues_count":0,"forks_count":0,"subscribers_count":6,"default_branch":"master","last_synced_at":"2025-02-14T08:47:04.967Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":null,"language":"Rust","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"other","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/JaneliaSciComp.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE.txt","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2023-03-07T18:01:15.000Z","updated_at":"2023-08-18T07:19:02.000Z","dependencies_parsed_at":null,"dependency_job_id":"d1eb5961-8ff5-480e-99ae-749c34a734b2","html_url":"https://github.com/JaneliaSciComp/aitch","commit_stats":null,"previous_names":[],"tags_count":7,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JaneliaSciComp%2Faitch","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JaneliaSciComp%2Faitch/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JaneliaSciComp%2Faitch/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/JaneliaSciComp%2Faitch/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/JaneliaSciComp","download_url":"https://codeload.github.com/JaneliaSciComp/aitch/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":239959949,"owners_count":19725221,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-11-08T14:23:24.925Z","updated_at":"2026-02-27T18:11:33.750Z","avatar_url":"https://github.com/JaneliaSciComp.png","language":"Rust","readme":"`aitch` (pronounced like the letter H) implements a light-weight cluster-style\nscheduler for use on a single workstation.  Unlike `parallel`, `xargs`, and\nother similar utilities, which presume that all jobs use the same amount\nof resources, `aitch` permits one to request a different amount from an\narbitrary number of unique kinds of resources.  In effect it implements a\ncounting semaphore for each resource and provides a means to adjust their\nvalues by non-negative integers.\n\n# Installation #\n\nInstall rust as described [here](https://www.rust-lang.org/).  Build with\n`cargo build`.\n\n# Basic Usage #\n\nSay for example your machine has six cores and you want to batch a heterogeneous\ncollection of jobs which require different numbers of cores:\n\n```\nhstart 6\nhsubmit 2 my-two-slot-job\nhsubmit 4 my-four-slot-job [and-its-args...]\nhsubmit 1 my-single-slot-job\n```\n\nIn this case, the third job is queued and remains pending until one of the\nfirst two finish.\n\nOther kinds of resources you might want to meter out include accelerator\ncards, memory, and file or network I/O.  So for example, if in addition\nyour machine also has two GPUs and 32 GB of RAM, the scheduler could be\nconfigured as follows:\n\n```\nhstart 6,2,32\nhsubmit 1,0,32 my-high-memory-app\nhsubmit 2,1,8 my-visualization-app\nhsubmit 6,0,1 my-compute-intensive-app\nhsubmit 0,0,0 'touch helloworld'\n```\n\nShould your jobs need to know which specific slot of a resource they consume,\nan environment variable prefixed with \"QUEUE\" is defined:\n\n```\necho \"export CUDA_VISIBLE_DEVICES=$QUEUE1; \\\n      python -m 'import tensorflow...'\" \u003e my-deeplearning-app\nhsubmit 2,1,4 my-deeplearning-app\n```\n\nIn this case, QUEUE1 would be set to either 0 or 1, assuming the `hstart`\nfrom above was still in effect.  QUEUE0 would similarly be two integers\nbetween 0 and 5 separated by a comma.\n\nOne can also specify job dependencies:\n\n```\ndep1=`hsubmit 6,0,12 do-me-first`\nhsubmit 1,0,1 --dep $dep1 wait-for-do-me-first-to-finish\n```\n\nand pass in environment variables:\n\n```\nhsubmit 1,0,0 --env FOO=bar winnie-the-pooh\n```\n\nand redirect the standard streams:\n\n```\nhsubmit 1,0,3 --out stdout.txt -err stderr.txt log-the-results\n```\n\nBesides the `hstart` and `hsubmit` commands, there are also `hjobs`, `hkill`,\n`hnslots`, `hstatus`, and `hstop`.  Usage information for each is displayed\nwith the `--help` flag.\n\n# Development #\n\nRun the tests with `cargo test`.\n\nBuild and test on a local disk with the `--target-dir \u003cDIR\u003e` flag.  Useful if\nthe source is on a networked fileshare which doesn't support locking.\n\nUpdate rust and cargo with `rustup update`.\n\nThe conda-forge feedstock is at https://github.com/conda-forge/aitch-feedstock.\n\nCheck the formatting of the feedstock recipe with `conda smithy recipe-lint`.\n\nGenerate a new key for the source code archive with `openssl sha256 \u003cgithub-tar-gz-file\u003e`.\n","funding_links":[],"categories":[],"sub_categories":[],"project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjaneliascicomp%2Faitch","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fjaneliascicomp%2Faitch","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fjaneliascicomp%2Faitch/lists"}