{"id":13602836,"url":"https://github.com/binpash/pash","last_synced_at":"2026-01-22T17:36:49.810Z","repository":{"id":37022179,"uuid":"194473604","full_name":"binpash/pash","owner":"binpash","description":"PaSh: Light-touch Data-Parallel Shell Processing","archived":false,"fork":false,"pushed_at":"2025-04-14T18:44:22.000Z","size":86879,"stargazers_count":572,"open_issues_count":61,"forks_count":44,"subscribers_count":13,"default_branch":"main","last_synced_at":"2025-05-07T22:46:45.645Z","etag":null,"topics":["bash","bash-scripting","data-analysis","parallelism","pash","posix-sh","shell"],"latest_commit_sha":null,"homepage":"","language":"Shell","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"mit","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/binpash.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":"docs/contributing/contrib.md","funding":null,"license":"LICENSE","code_of_conduct":"CODE_OF_CONDUCT.md","threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null,"zenodo":null}},"created_at":"2019-06-30T04:12:11.000Z","updated_at":"2025-05-04T00:17:23.000Z","dependencies_parsed_at":"2024-02-01T06:27:30.472Z","dependency_job_id":"5a02042d-4e9c-4560-9d1f-0b68c0df0645","html_url":"https://github.com/binpash/pash","commit_stats":{"total_commits":2788,"total_committers":34,"mean_commits":82.0,"dds":0.4415351506456241,"last_synced_commit":"0d0a563b17c23cdf2d4f8228da0439bf3eaed26a"},"previous_names":[],"tags_count":19,"template":false,"template_full_name":null,"repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/binpash%2Fpash","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/binpash%2Fpash/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/binpash%2Fpash/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/binpash%2Fpash/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/binpash","download_url":"https://codeload.github.com/binpash/pash/tar.gz/refs/heads/main","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":254319718,"owners_count":22051072,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["bash","bash-scripting","data-analysis","parallelism","pash","posix-sh","shell"],"created_at":"2024-08-01T18:01:39.497Z","updated_at":"2026-01-22T17:36:49.805Z","avatar_url":"https://github.com/binpash.png","language":"Shell","funding_links":[],"categories":["bash"],"sub_categories":[],"readme":"## PaSh: Light-touch Data-Parallel Shell Processing\n\n\u003e _A system for parallelizing POSIX shell scripts._\n\u003e _Hosted by the [Linux Foundation](https://linuxfoundation.org/press-release/linux-foundation-to-host-the-pash-project-accelerating-shell-scripting-with-automated-parallelization-for-industrial-use-cases/)._\n\n[![Tests](https://github.com/binpash/pash/actions/workflows/tests.yaml/badge.svg?branch=main\u0026event=push)](https://github.com/binpash/pash/actions/workflows/tests.yaml?query=branch%3Amain++)\n[![Build](https://github.com/binpash/pash/actions/workflows/build.yaml/badge.svg?branch=main\u0026event=push)](https://github.com/binpash/pash/actions/workflows/build.yaml?query=branch%3Amain++)\n\nQuick Jump: [Running PaSh](#running-pash) | [Installation](#installation) | [Testing](#testing) | [Repo Structure](#repo-structure) | [Community \u0026 More](#community--more) | [Citing](#citing)\n\n\n## Installation via pip (Recommended)\n```sh\npip install binpash-pash\npash -c \"echo hi\"\n```\n\n## Running PaSh\n\nTo parallelize, say, `./evaluation/intro/hello-world.sh` with parallelization degree of 2× run:\n\n```sh\npash -w 2 ./evaluation/intro/hello-world.sh\n```\n\nIf the script contains bash specific syntax, add the beta `--bash` flag to enable support.\n\nRun `pash --help` to get more information about the available commands.\nJump to [docs/tutorial](docs/tutorial/) for a longer tutorial.\n\n\n## Local testing (development)\n\nTo install and run PaSh for local development:\n\n```sh\npip install -e .\npash --help\npash -c \"echo hello | cat\"\n./scripts/run_tests.sh\n```\n\nFor more details, manual installation, or other platforms see [installation instructions](./docs/install).\n\n## Running with a local annotations library\n\nTo run with a local version of the library, please refer to the documentation [local annotations setup and usage](docs/local-annotations-library-documentation.md) \n\n## Repo Structure\n\nThis repo hosts the core `pash` development. The structure is as follows:\n\n* [src/pash/](./src/pash/): Main Python package (installed via pip)\n  * [preprocessor](./src/pash/preprocessor): Parses shell scripts, expands variables, and identifies dataflow regions for compilation.\n  * [compiler](./src/pash/compiler): Translates shell dataflow regions to IRs and applies parallelization transformations.\n  * [jit_runtime](./src/pash/jit_runtime): Just-in-time runtime that executes compiled regions and manages shell state.\n  * [runtime](./src/pash/runtime): Runtime components — e.g., `eager`, `split`, and associated combiners.\n* [docs](./docs): Design documents, tutorials, installation instructions, etc.\n* [evaluation](./evaluation): Shell pipelines and example [scripts](./evaluation/other/more-scripts) used for the evaluation.\n* [scripts](./scripts): Scripts related to continuous integration, deployment, and testing.\n\n## Community \u0026 More\n\nChat:\n* [Discord Server](ttps://discord.com/channels/947328962739187753/) ([Invite](https://discord.gg/6vS9TB97be))\n\nMailing Lists:\n* [pash-devs](https://groups.google.com/g/pash-devs): Join this mailing list for discussing all things `pash`\n* [pash-commits](https://groups.google.com/g/pash-commits): Join this mailing list for commit notifications\n\nDevelopment/contributions:\n* Contribution guide: [docs/contributing](docs/contributing/contrib.md)\n* Continuous Integration Server: [ci.binpa.sh](http://ci.binpa.sh)\n\n## Citing\n\nIf you used PaSh, consider citing the following paper:\n```\n@inproceedings{pash2021eurosys,\nauthor = {Vasilakis, Nikos and Kallas, Konstantinos and Mamouras, Konstantinos and Benetopoulos, Achilles and Cvetkovi\\'{c}, Lazar},\ntitle = {PaSh: Light-Touch Data-Parallel Shell Processing},\nyear = {2021},\nisbn = {9781450383349},\npublisher = {Association for Computing Machinery},\naddress = {New York, NY, USA},\nurl = {https://doi.org/10.1145/3447786.3456228},\ndoi = {10.1145/3447786.3456228},\npages = {49–66},\nnumpages = {18},\nkeywords = {POSIX, Unix, pipelines, automatic parallelization, source-to-source compiler, shell},\nlocation = {Online Event, United Kingdom},\nseries = {EuroSys '21}\n}\n```\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbinpash%2Fpash","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Fbinpash%2Fpash","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Fbinpash%2Fpash/lists"}