{"id":25451272,"url":"https://github.com/lherman-cs/gotopus","last_synced_at":"2026-02-15T23:03:51.668Z","repository":{"id":57522244,"uuid":"253022554","full_name":"lherman-cs/gotopus","owner":"lherman-cs","description":"Gotopus is a minimalistic tool that runs arbitrary commands concurrently.","archived":false,"fork":false,"pushed_at":"2020-04-16T01:39:13.000Z","size":61,"stargazers_count":15,"open_issues_count":0,"forks_count":0,"subscribers_count":2,"default_branch":"master","last_synced_at":"2025-04-08T22:23:14.201Z","etag":null,"topics":["build","concurrency","dependency-graph","docker"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/lherman-cs.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null}},"created_at":"2020-04-04T14:47:01.000Z","updated_at":"2024-09-27T11:46:42.000Z","dependencies_parsed_at":"2022-08-26T23:41:50.101Z","dependency_job_id":null,"html_url":"https://github.com/lherman-cs/gotopus","commit_stats":null,"previous_names":[],"tags_count":36,"template":false,"template_full_name":null,"purl":"pkg:github/lherman-cs/gotopus","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lherman-cs%2Fgotopus","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lherman-cs%2Fgotopus/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lherman-cs%2Fgotopus/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lherman-cs%2Fgotopus/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/lherman-cs","download_url":"https://codeload.github.com/lherman-cs/gotopus/tar.gz/refs/heads/master","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/lherman-cs%2Fgotopus/sbom","host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":267962235,"owners_count":24172609,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2022-07-04T15:15:14.044Z","status":"online","status_checked_at":"2025-07-30T02:00:09.044Z","response_time":70,"last_error":null,"robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":true,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["build","concurrency","dependency-graph","docker"],"created_at":"2025-02-17T22:24:41.722Z","updated_at":"2026-02-15T23:03:46.640Z","avatar_url":"https://github.com/lherman-cs.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"# gotopus\n\n[![codecov](https://codecov.io/gh/lherman-cs/gotopus/branch/master/graph/badge.svg)](https://codecov.io/gh/lherman-cs/gotopus)\n[![Go Report Card](https://goreportcard.com/badge/github.com/lherman-cs/gotopus)](https://goreportcard.com/report/github.com/lherman-cs/gotopus)\n[![Codacy Badge](https://api.codacy.com/project/badge/Grade/81a94fecb21b45bcb89ef6b8c6e3b682)](https://www.codacy.com/manual/lherman-cs/gotopus?utm_source=github.com\u0026amp;utm_medium=referral\u0026amp;utm_content=lherman-cs/gotopus\u0026amp;utm_campaign=Badge_Grade)\n[![License](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)\n\nGotopus is a minimalistic tool that runs arbitrary commands concurrently. You define your commands with their dependencies and Gotopus will take care of the rest, running them concurrently when possible.\n\n## Contents\n- [Features](#features)\n- [Installation](#installation)\n- [Getting Started](#getting-started)\n  - [Basic Usage](#basic-usage)\n  - [Environment Variables](#environment-variables)\n  - [Concurrency vs Parallelism](#concurrency-vs-parallelism)\n- [FAQ](#faq)\n  - [Why does the config format look similar to Github Actions](#why-does-the-config-format-look-similar-to-github-actions)\n\n## Features\n\n- [X] Concurrently run steps, speeding up running time\n- [X] Local or remote configs\n- [X] Easy to install\n- [X] Circular dependency detection\n- [X] Clean step definition with [YAML](https://en.wikipedia.org/wiki/YAML)\n- [X] [Builtin and user environment variables](#environment-variables)\n\n## Installation\n\n```sh\ncurl -sf https://gobinaries.com/lherman-cs/gotopus | sh\n```\n\n## Getting Started\n\n### Basic Usage\n\n```\nUsage: gotopus \u003curl or filepath\u003e ...\n\n  -max_workers uint\n    \tlimits the number of workers that can run concurrently (default 0 or limitless)\n```\n\n```yaml\n# examples/basic.yaml\njobs:\n  job1:\n    steps:\n      - run: sleep 1 \u0026\u0026 echo \"job1\"\n  job2:\n    needs:\n      - job1\n    steps:\n      - run: echo \"job2\"\n  job3:\n    steps:\n      - run: echo \"job3\"\n```\n\nTo use `basic.yaml` above, you can run the following command:\n\n```sh\ngotopus basic.yaml\n```\n\nOr you can simply give a URL to this file:\n\n```sh\ngotopus https://raw.githubusercontent.com/lherman-cs/gotopus/master/examples/basic.yaml\n```\n\nBy default, if you don't set `max_workers` to any number greater than 0, gotopus will create a pool of workers without a limit in lazy way. From the example above, 2 workers will be allocated instead of 3. The process looks like following:\n\n```\njob1 gets scheduled\nspawn worker #0\nworker #0 executes job1\njob3 gets scheduled\nspawn worker #1\nworker #1 executes job3\njob3 finishes\njob1 finishes\neither worker #0 or #1 executes job2\njob2 finishes\n```\n\n### Environment Variables\nWhenever a step runs, there are 3 kinds of environments that are going to be set and they'll have the priority order (in case of a conflict happens, the higher priority environment variable will be chosen) as listed below, where user environment variables will have the highest priority:\n\n* User: these environment variables are defined by the user in yaml in each step.\n* Builtin: environment variables that come from gotopus and they'll be prefixed with `GOTOPUS_`.\n\n  * `GOTOPUS_JOB_ID`\n  * `GOTOPUS_JOB_NAME`\n  * `GOTOPUS_STEP_NAME`\n  * `GOTOPUS_WORKER_ID`\n\n* System: inherits all the environments variables from the system when you run gotopus.\n\nFollowing is an example how you define and use environment variables:\n\n```yaml\n# examples/env.yaml\njobs:\n  job:\n    steps:\n      - name: Install dependencies\n        run: echo \"$GOTOPUS_STEP_NAME\"\n      - run: echo \"$name\"\n        env:\n          name: Lukas Herman\n```\n\n### Concurrency vs Parallelism\nLet's imagine that there are 2 commands that we want to execute:\n\n1. First command: `sleep 2 \u0026\u0026 echo \"job 1\"`\n2. Second command: `sleep 3 \u0026\u0026 echo \"job 2\"`\n\nNormally, this would take **5 seconds** to finish since you need to run them sequentially. But, if you run this concurrently, this would take **~3 seconds even when you only have 1 CPU core!**. This is possible because they run **conccurrently NOT in parallel**. For more information, there's this awesome video from Rob Pike that specfically talks about \"Concurency Is Not Parallelism\".\n\n[![Conccurency vs Parallelism](https://img.youtube.com/vi/cN_DpYBzKso/0.jpg)](https://www.youtube.com/watch?v=cN_DpYBzKso)\n\n```sh\ntime gotopus https://raw.githubusercontent.com/lherman-cs/gotopus/master/examples/concurrency.yaml\n```\n\n## FAQ\n\n### Why does the config format look similar to Github Actions?\nThis is intentional because I think Github Actions format is pretty clean and also one of my inspirations to create Gotopus. Although Gotopus and Github Actions seem to have similar functionalities, you can definitelly use them together! For example, you can run steps of a job concurrently with Gotopus (running steps concurrently is not supported currently: https://github.community/t5/GitHub-Actions/Steps-in-parallel/td-p/32635).\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flherman-cs%2Fgotopus","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Flherman-cs%2Fgotopus","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Flherman-cs%2Fgotopus/lists"}