{"id":44801814,"url":"https://github.com/tednaleid/ganda","last_synced_at":"2026-02-16T13:34:15.010Z","repository":{"id":41551057,"uuid":"79582455","full_name":"tednaleid/ganda","owner":"tednaleid","description":"fast cmd-line app that quickly requests millions of urls and can save/echo the results","archived":false,"fork":false,"pushed_at":"2024-11-03T22:12:09.000Z","size":2958,"stargazers_count":62,"open_issues_count":2,"forks_count":8,"subscribers_count":4,"default_branch":"main","last_synced_at":"2024-11-03T23:18:45.532Z","etag":null,"topics":["command-line","curl","golang","http","http-client","https","linux","macos","pipe","shell","web-scraping"],"latest_commit_sha":null,"homepage":"","language":"Go","has_issues":true,"has_wiki":null,"has_pages":null,"mirror_url":null,"source_name":null,"license":"apache-2.0","status":null,"scm":"git","pull_requests_enabled":true,"icon_url":"https://github.com/tednaleid.png","metadata":{"files":{"readme":"README.md","changelog":null,"contributing":null,"funding":null,"license":"LICENSE","code_of_conduct":null,"threat_model":null,"audit":null,"citation":null,"codeowners":null,"security":null,"support":null,"governance":null,"roadmap":null,"authors":null,"dei":null,"publiccode":null,"codemeta":null}},"created_at":"2017-01-20T17:48:45.000Z","updated_at":"2024-11-03T22:12:13.000Z","dependencies_parsed_at":"2024-05-04T21:23:28.332Z","dependency_job_id":"2fbc7dd2-af78-4710-b360-c93f4b986f60","html_url":"https://github.com/tednaleid/ganda","commit_stats":{"total_commits":45,"total_committers":3,"mean_commits":15.0,"dds":0.0444444444444444,"last_synced_commit":"ba23a9da51055d5bec2175e7050280ea7772d48e"},"previous_names":[],"tags_count":22,"template":false,"template_full_name":null,"purl":"pkg:github/tednaleid/ganda","repository_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tednaleid%2Fganda","tags_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tednaleid%2Fganda/tags","releases_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tednaleid%2Fganda/releases","manifests_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tednaleid%2Fganda/manifests","owner_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners/tednaleid","download_url":"https://codeload.github.com/tednaleid/ganda/tar.gz/refs/heads/main","sbom_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/tednaleid%2Fganda/sbom","scorecard":{"id":872505,"data":{"date":"2025-08-11","repo":{"name":"github.com/tednaleid/ganda","commit":"54ceb8b88b20a3a2d4bf7c43a542a82e2772aa2a"},"scorecard":{"version":"v5.2.1-40-gf6ed084d","commit":"f6ed084d17c9236477efd66e5b258b9d4cc7b389"},"score":1.7,"checks":[{"name":"Token-Permissions","score":-1,"reason":"No tokens found","details":null,"documentation":{"short":"Determines if the project's workflows follow the principle of least privilege.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#token-permissions"}},{"name":"Binary-Artifacts","score":10,"reason":"no binaries found in the repo","details":null,"documentation":{"short":"Determines if the project has generated executable (binary) artifacts in the source repository.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#binary-artifacts"}},{"name":"Maintained","score":0,"reason":"0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0","details":null,"documentation":{"short":"Determines if the project is \"actively maintained\".","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#maintained"}},{"name":"Dangerous-Workflow","score":-1,"reason":"no workflows found","details":null,"documentation":{"short":"Determines if the project's GitHub Action workflows avoid dangerous patterns.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#dangerous-workflow"}},{"name":"Pinned-Dependencies","score":-1,"reason":"no dependencies found","details":null,"documentation":{"short":"Determines if the project has declared and pinned the dependencies of its build process.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#pinned-dependencies"}},{"name":"Code-Review","score":1,"reason":"Found 1/10 approved changesets -- score normalized to 1","details":null,"documentation":{"short":"Determines if the project requires human code review before pull requests (aka merge requests) are merged.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#code-review"}},{"name":"Packaging","score":-1,"reason":"packaging workflow not detected","details":["Warn: no GitHub/GitLab publishing workflow detected."],"documentation":{"short":"Determines if the project is published as a package that others can easily download, install, easily update, and uninstall.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#packaging"}},{"name":"CII-Best-Practices","score":0,"reason":"no effort to earn an OpenSSF best practices badge detected","details":null,"documentation":{"short":"Determines if the project has an OpenSSF (formerly CII) Best Practices Badge.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#cii-best-practices"}},{"name":"Security-Policy","score":0,"reason":"security policy file not detected","details":["Warn: no security policy file detected","Warn: no security file to analyze","Warn: no security file to analyze","Warn: no security file to analyze"],"documentation":{"short":"Determines if the project has published a security policy.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#security-policy"}},{"name":"Fuzzing","score":0,"reason":"project is not fuzzed","details":["Warn: no fuzzer integrations found"],"documentation":{"short":"Determines if the project uses fuzzing.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#fuzzing"}},{"name":"License","score":10,"reason":"license file detected","details":["Info: project has a license file: LICENSE:0","Info: FSF or OSI recognized license: Apache License 2.0: LICENSE:0"],"documentation":{"short":"Determines if the project has defined a license.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#license"}},{"name":"Branch-Protection","score":0,"reason":"branch protection not enabled on development/release branches","details":["Warn: branch protection not enabled for branch 'main'"],"documentation":{"short":"Determines if the default and release branches are protected with GitHub's branch protection settings.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#branch-protection"}},{"name":"Signed-Releases","score":0,"reason":"Project has not signed or included provenance with any releases.","details":["Warn: release artifact v1.0.3 not signed: https://api.github.com/repos/tednaleid/ganda/releases/167943746","Warn: release artifact v1.0.2 not signed: https://api.github.com/repos/tednaleid/ganda/releases/167895832","Warn: release artifact v1.0.1 not signed: https://api.github.com/repos/tednaleid/ganda/releases/167737039","Warn: release artifact v1.0.0 not signed: https://api.github.com/repos/tednaleid/ganda/releases/167726311","Warn: release artifact v0.1.10 not signed: https://api.github.com/repos/tednaleid/ganda/releases/55786697","Warn: release artifact v1.0.3 does not have provenance: https://api.github.com/repos/tednaleid/ganda/releases/167943746","Warn: release artifact v1.0.2 does not have provenance: https://api.github.com/repos/tednaleid/ganda/releases/167895832","Warn: release artifact v1.0.1 does not have provenance: https://api.github.com/repos/tednaleid/ganda/releases/167737039","Warn: release artifact v1.0.0 does not have provenance: https://api.github.com/repos/tednaleid/ganda/releases/167726311","Warn: release artifact v0.1.10 does not have provenance: https://api.github.com/repos/tednaleid/ganda/releases/55786697"],"documentation":{"short":"Determines if the project cryptographically signs release artifacts.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#signed-releases"}},{"name":"SAST","score":0,"reason":"SAST tool is not run on all commits -- score normalized to 0","details":["Warn: 0 commits out of 25 are checked with a SAST tool"],"documentation":{"short":"Determines if the project uses static code analysis.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#sast"}},{"name":"Vulnerabilities","score":0,"reason":"14 existing vulnerabilities detected","details":["Warn: Project is vulnerable to: GHSA-29gw-9793-fvw7","Warn: Project is vulnerable to: PYSEC-2015-24 / GHSA-4vwq-x64q-j4cj","Warn: Project is vulnerable to: PYSEC-2017-46 / GHSA-66gw-5xpf-gfp5","Warn: Project is vulnerable to: PYSEC-2015-25 / GHSA-92mr-v722-f48m","Warn: Project is vulnerable to: PYSEC-2022-12 / GHSA-pq7m-3gw7-gq5x","Warn: Project is vulnerable to: PYSEC-2017-47","Warn: Project is vulnerable to: GHSA-33p9-3p43-82vq","Warn: Project is vulnerable to: PYSEC-2022-42974 / GHSA-m678-f26j-3hrp","Warn: Project is vulnerable to: GO-2025-3553 / GHSA-mh63-6h87-95cp","Warn: Project is vulnerable to: GO-2024-3321 / GHSA-v778-237x-gjrc","Warn: Project is vulnerable to: GO-2025-3487 / GHSA-hcg3-q754-cr77","Warn: Project is vulnerable to: GO-2024-3333","Warn: Project is vulnerable to: GO-2025-3503 / GHSA-qxp5-gwg8-xv66","Warn: Project is vulnerable to: GO-2025-3595 / GHSA-vvgc-356p-c3xw"],"documentation":{"short":"Determines if the project has open, known unfixed vulnerabilities.","url":"https://github.com/ossf/scorecard/blob/f6ed084d17c9236477efd66e5b258b9d4cc7b389/docs/checks.md#vulnerabilities"}}]},"last_synced_at":"2025-08-24T04:42:56.114Z","repository_id":41551057,"created_at":"2025-08-24T04:42:56.114Z","updated_at":"2025-08-24T04:42:56.114Z"},"host":{"name":"GitHub","url":"https://github.com","kind":"github","repositories_count":286080680,"owners_count":29509176,"icon_url":"https://github.com/github.png","version":null,"created_at":"2022-05-30T11:31:42.601Z","updated_at":"2026-02-16T09:05:14.864Z","status":"ssl_error","status_checked_at":"2026-02-16T08:55:59.364Z","response_time":115,"last_error":"SSL_read: unexpected eof while reading","robots_txt_status":"success","robots_txt_updated_at":"2025-07-24T06:49:26.215Z","robots_txt_url":"https://github.com/robots.txt","online":false,"can_crawl_api":true,"host_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub","repositories_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories","repository_names_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/repository_names","owners_url":"https://repos.ecosyste.ms/api/v1/hosts/GitHub/owners"}},"keywords":["command-line","curl","golang","http","http-client","https","linux","macos","pipe","shell","web-scraping"],"created_at":"2026-02-16T13:34:14.219Z","updated_at":"2026-02-16T13:34:15.001Z","avatar_url":"https://github.com/tednaleid.png","language":"Go","funding_links":[],"categories":[],"sub_categories":[],"readme":"\n# ganda - High-Performance HTTP Request CLI\n\n## Overview\n\n`ganda` lets you make HTTP/HTTPS requests to hundreds to millions of URLs in just a few minutes.\nIt's designed with the Unix philosophy of [\"do one thing well\"](https://en.wikipedia.org/wiki/Unix_philosophy#Do_One_Thing_and_Do_It_Well) and wants to be used in a chain of command line pipes to make its requests in parallel. \nBy default, it will echo all response bodies to standard out but can optionally save the results of each request in a directory for later analysis.\n\n### Key Features\n\n* **Parallel Request Processing:** Handle thousands of URLs simultaneously with customizable worker counts.\n* **Flexible Output Options:** Output responses to stdout, save to a directory, or format as JSON for easy parsing.\n* **Integrate with CLI Tools:** Works well with tools like jq, awk, sort, and more for powerful data transformations.\n\n### Why use `ganda` over `curl` (or `wget`, `httpie`, `postman-cli`, ...)?\n\nAll existing CLI tools for making HTTP requests are oriented around making a single request at a time.  They're great\nat starting a pipe of commands (ex: `curl \u003curl\u003e | jq .`) but they're awkward to use beyond a few requests.\n\nThe easiest way to use them is in a bash `for` loop or with something like `xargs`.  This is slow and expensive as they open up a new HTTP connection on every request.\n\n`ganda` makes many requests in parallel and can maintain context between the request and response.  It's designed to\nbe used in a pipeline of commands and can be used to make hundreds of thousands of requests in just a few minutes.\n\n`ganda` will reuse HTTP connections and can specify how many \"worker\" threads should be used to tightly control parallelism.\n\nThe closest CLIs I've found to `ganda` are load-testing tools like `vegeta`.  They're able to make many requests in\nparallel, but they're not designed to only call each URL once, don't maintain context between the request and response,\nand don't have the same flexibility in how the response is handled.\n\n`ganda` isn't for load testing, it's for making lots of requests in parallel and processing the results in a pipeline.\n\n## Documentation Links\n\n* [Installation](#installation)\n* [Usage Configuration Options](#usage--configuration-options)\n* [Quick Examples](#quick-examples)\n* [Advanced Use Cases](#sample-advanced-use-cases)\n\n# Installation\n\nOne currently has 3 options:\n\n1\\. On MacOS you can install using [homebrew](https://brew.sh/)\n```bash\nbrew tap tednaleid/homebrew-ganda\nbrew install ganda\n```\n\n2\\. Download the appropriate binary from the [releases page]((https://github.com/tednaleid/ganda/releases) and put it in your path\n\n3\\. Compile from source with golang:\n\n```bash\ngo install github.com/tednaleid/ganda@latest\n```\n\nor, if you have this repo downloaded locally:\n\n```bash\nmake install\n```\n\nto install in your `$GOPATH/bin` (which you want in your `$PATH`)\n\n# Usage \u0026 Configuration Options\n\n```bash\nganda help\n\nNAME:\nganda - make http requests in parallel\n\nUSAGE:\n\u003curls/requests on stdout\u003e | ganda [options]\n\nVERSION:\n   1.0.2\n\nDESCRIPTION:\n   Pipe urls to ganda over stdout to make http requests to each url in parallel.\n\nAUTHOR:\n   Ted Naleid \u003ccontact@naleid.com\u003e\n\nCOMMANDS:\n   echoserver  Starts an echo server, --port \u003cport\u003e to override the default port of 8080\n   help, h     Shows a list of commands or help for one command\n\nGLOBAL OPTIONS:\n   --base-retry-millis value                              the base number of milliseconds to wait before retrying a request, exponential backoff is used for retries (default: 1000)\n   --response-body value, -B value                        transforms the body of the response. Values: 'raw' (unchanged), 'base64', 'discard' (don't emit body), 'escaped' (JSON escaped string), 'sha256' (default: raw)\n   --connect-timeout-millis value                         number of milliseconds to wait for a connection to be established before timeout (default: 10000)\n   --header value, -H value [ --header value, -H value ]  headers to send with every request, can be used multiple times (gzip and keep-alive are already there)\n   --insecure, -k                                         if flag is present, skip verification of https certificates (default: false)\n   --json-envelope, -J                                    emit result with JSON envelope with url, status, length, and body fields, assumes result is valid json (default: false)\n   --color                                                if flag is present, add color to success/warn messages (default: false)\n   --output-directory value                               if flag is present, save response bodies to files in the specified directory\n   --request value, -X value                              HTTP request method to use (default: \"GET\")\n   --retry value                                          max number of retries on transient errors (5XX status codes/timeouts) to attempt (default: 0)\n   --silent, -s                                           if flag is present, omit showing response code for each url only output response bodies (default: false)\n   --subdir-length value                                  length of hashed subdirectory name to put saved files when using --output-directory; use 2 for \u003e 5k urls, 4 for \u003e 5M urls (default: 0)\n   --throttle-per-second value                            max number of requests to process per second, default is unlimited (default: -1)\n   --workers value, -W value                              number of concurrent workers that will be making requests, increase this for more requests in parallel (default: 1)\n   --help, -h                                             show help (default: false)\n   --version, -v                                          print the version (default: false)\n```\n\n# Quick Examples\n\nHere are a few quick examples to show how `ganda` can be used.\n\n### Example 1: Basic Request from a List of IDs\n\nGiven a file with a list of IDs in it, you could do something like:\n\n```bash\ncat id_list.txt | awk '{printf \"https://api.example.com/resource/%s?key=foo\\n\", $1}' | ganda\n```\nand that will pipe a stream of URLs into `ganda` in the format `https://api.example.com/resource/\u003cID\u003e?key=foo`.\n\nThis command:\n* Reads IDs from `id_list.txt`. \n* Uses `awk` to format each ID as a URL. \n* Pipes the generated URLs into `ganda` for parallel requests.\n\n### Example 2: Requesting URLs from a File\n\nIf you have a file containing URLs (one per line), you can pass it directly to `ganda`:\n\n```bash\nganda my_file_of_urls.txt\n```\nThis command sends each URL in `my_file_of_urls.txt` as a request in parallel. You can control the output location by specifying an output directory with `-o \u003cdirectory\u003e`.\n\n### Example 3: Save Responses to a Directory\n\nTo save each response in a separate file within a specified directory:\n\n```bash\ncat urls.txt | ganda -o response_dir\n```\n\nTo save all responses to a single file, you can use standard output redirection:\n\n```bash\ncat urls.txt | ganda \u003e results.txt\n```\n\nFor many more examples, take a look at the [Tour of `ganda`](docs/GANDA_TOUR.ipynb).\n\n## Sample Advanced Use Cases\n\n`ganda` enables powerful workflows that would otherwise require custom scripting. Here are a few advanced examples.\n\n### Example 1: Consuming Events from Kafka and Calling an API\n\nUsing `kcat` (https://github.com/edenhill/kcat) (or another Kafka CLI that emits events from Kafka topics), we can consume all the events on a Kafka topic, then use `jq` to pull an identifier out of an event and make an API call for every identifier:\n\n```bash\n# get all events on the `my-topic` topic\nkcat -C -e -q -b broker.example.com:9092 -t my-topic |\\\n  # parse the identifier out of the JSON event\n  jq -r '.identifier' |\\\n  # use awk to turn that identifier into an URL\n  awk '{ printf \"https://api.example.com/item/%s\\n\", $1}' |\\\n  # have 5 workers make requests and use a static header with and API key for every request\n  ganda -s -W 5 -H \"X-Api-Key: my-key\" |\\\n  # parse the `value` out of the response and emit it on stdout\n  jq -r '.value'\n```\n\n### Example 2: Requesting Multiple Pages from an API\n\nHere, we ask for the first 100 pages from an API.  Each returns a JSON list of `status` fields.  Pull those `status` fields out and do a unique count on the distribution.\n\n```bash\n# emit a sequence of the numbers from 1 to 100\nseq 100 |\\\n  # use awk to create an url asking for each of the buckets\n  awk '{printf \"https://example.com/items?type=BUCKET\u0026value=%s\\n\", $1}' |\\\n  # use a single ganda worker to ask for each page in sequence\n  ganda -s -W 1 -H \"X-Api-Key: my-key\" |\\\n  # use jq to parse the resulting json and grab the status\n  jq -r '.items[].status' |\\ \n  sort |\\\n  # get a unique count of how many times each status appears\n  uniq -c\n\n  41128 DELETED\n   6491 INITIATED\n  34222 PROCESSED\n   5032 ERRORED\n```\n## Contribution Guidelines\n\nIf you like to contribute, please follow these steps:\n1. Fork the repository and create a new branch.\n2. Make your changes and write tests if applicable.\n3. Submit a pull request with a clear description of your changes.\n","project_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftednaleid%2Fganda","html_url":"https://awesome.ecosyste.ms/projects/github.com%2Ftednaleid%2Fganda","lists_url":"https://awesome.ecosyste.ms/api/v1/projects/github.com%2Ftednaleid%2Fganda/lists"}