{"id":16956375,"url":"https://github.com/bgamari/tpar","last_synced_at":"2025-04-11T21:40:43.853Z","repository":{"id":11896077,"uuid":"14459197","full_name":"bgamari/tpar","owner":"bgamari","description":"A simple distributed parallel job execution engine in Haskell","archived":false,"fork":false,"pushed_at":"2016-08-06T19:47:09.000Z","size":120,"stargazers_count":7,"open_issues_count":3,"forks_count":1,"subscribers_count":4,"default_branch":"master","last_synced_at":"2025-04-06T13:17:24.841Z","etag":null,"topics":[],"latest_commit_sha":null,"homepage":"","language":"Haskell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"bsd-3-clause","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/bgamari.png","metadata":{"files":{"readme":"README.mkd","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2013-11-17T01:32:44.000Z","updated_at":"2024-01-25T08:36:38.000Z","dependencies_parsed_at":"2022-09-07T02:50:42.151Z","dependency_job_id":null,"html_url":"https://github.com/bgamari/tpar","commit_stats":null,"previous_names":[],"tags_count":0,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bgamari%2Ftpar","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bgamari%2Ftpar/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bgamari%2Ftpar/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/bgamari%2Ftpar/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/bgamari","download_url":"https://codeload.github.com/bgamari/tpar/tar.gz/refs/heads/master","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":248486240,"owners_count":21112036,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":[],"created_at":"2024-10-13T22:14:47.306Z","updated_at":"2025-04-11T21:40:43.829Z","avatar_url":"https://github.com/bgamari.png","language":"Haskell","funding_links":[],"categories":[],"sub_categories":[],"readme":"# tpar — simple parallel job scheduling\n\n`tpar` is a simple tool for concurrent job scheduling. Say you have a\ndirectory full of files which need processing,\n\n```bash\n$ ls\nfile1    file2     file3     file4    file5\n...\n```\n\nUsually one could use a `bash` for loop,\n\n```bash\n$ for f in *; do process $f; done;\n```\n\nBut if `process` is a long-running task and you have many cores at\nyour disposal, it would be nice to speed things up a bit,\n\n```bash\n$ tpar server -N8\n$ for f in *; do tpar enqueue -- process $f; done;\n```\n\nIf you have multiple machines with the data mounted over, say, NFS, they can\nalso help with churning through the queue,\n\n```bash\n$ for m in worker1 worker2 worker3; do\n\u003e   ssh $m -- tpar worker -H`hostname`;\n\u003e done\n```\n\n## Commands\n\n`tpar` has several subcommands,\n\n  * `tpar server` starts a local queue server.\n  * `tpar worker` starts a worker associated with\n    the given queue\n  * `tpar enqueue -- $cmd` enqueues a job in the given queue\n  * `tpar status` allows you to query for the status of the queue. You can also provide a job match expression \n  * `tpar kill` kills a running task (specified by a job match expression)\n  * `tpar watch` is analogous `tail -f`, watching the output of a set of running tasks\n  * `tpar dump` dumps a JSON representation of the queue state.\n\nNearly all of these commands will require that the `-H` option be provided\nspecifying the canonical hostname of the queue server (the machine running\n`tpar server`).\n\n\n## Job match expressions\n\nSeveral `tpar` commands accept a *job match expression*, which specifies the\nsubset of jobs on which the command should act. For instance (note the quotes to\nensure that `bash` doesn't interpret our symbols),\n\n```\n$ tpar status '*'    # This is equivalent to `tpar status` run without an argument\n$ tpar status id=2\n$ tpar status state=running\n$ tpar status 'name=\"my-job\" or name=\"my-other-job\"'\n```\n\nThese expressions consist of the primitive matches,\n * `name=\"STRING\"`, which matches on the job name provided in the `--name` of\n   `tpar enqueue`.\n * `id=`, which matches on the job ID\n * `state=`, which matches on the current state of the job (`queued`, `running`,\n   `finished`, `failed`, `killed`, or `code=N`)\n * `*`, which matches all jobs\n \nThese matches can be connected with the `and` and `or` operators, and inverted\nwith `!`.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbgamari%2Ftpar","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbgamari%2Ftpar","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbgamari%2Ftpar/lists"}